Creates a new web crawl data source and starts ingestion.
Required roles: All
API key for authentication
Name of the web crawl data source
200Start URL of the web crawl
2000Maximum crawl depth
1 <= x <= 10Maximum number of files to crawl
1 <= x <= 100000Path filters for crawling. The total number of characters across all elements in the array must be 2000 or fewer.
2000Content patterns for filtering. The total number of characters across all elements in the array must be 2000 or fewer.
2000When true, only HTML files will be downloaded
Whether to use a headless browser for crawling
File extensions to include (e.g. ".pdf", ".docx"). For supported file extensions, please refer to https://developer.qaip.com/docs/datasources#%E5%AF%BE%E5%BF%9C%E3%81%97%E3%81%A6%E3%81%84%E3%82%8B%E3%83%95%E3%82%A1%E3%82%A4%E3%83%AB%E5%BD%A2%E5%BC%8F
200010Recurrence rule (RFC 5545 RRULE)
Successfully created web crawl data source
Web crawl data source ID
Name of the web crawl ingestion setting
Start URL of the web crawl
Job status
unknown, queued, not_started, managed, starting, started, success, failure, canceling, canceled, deleting, delete_job_failure Web crawl ingestion setting ID
Creation time (Unix timestamp in seconds)
Job start time (Unix timestamp in seconds)
Job end time (Unix timestamp in seconds)