Create Web Crawl
POST
/
crawl
/
create
Copy
curl --request POST \
--url https://api.webtranspose.com/v1/crawl/create \
--header 'Content-Type: application/json' \
--data '{
"url": "<string>",
"render_js": false,
"max_pages": 123,
"allowed_urls": [],
"banned_urls": []
}'Assistant
Responses are generated using AI and may contain mistakes.
Body
application/json
Response
200
Successful operation
Copy
curl --request POST \
--url https://api.webtranspose.com/v1/crawl/create \
--header 'Content-Type: application/json' \
--data '{
"url": "<string>",
"render_js": false,
"max_pages": 123,
"allowed_urls": [],
"banned_urls": []
}'Webᵀ Crawl
- POSTCreate Web Crawl
- POSTStart / Resume Crawl
- POSTList Web Crawls
- POSTGet Crawl
- POSTDownload Crawl
- POSTGet Page (HTML, URLs, Text)
- POSTGet Page's Child URLs
- POSTSet Allowed Pages
- POSTSet Banned Pages
- POSTSet Max Pages
- POSTGet Visited Pages
- POSTGet Ignored Pages
- POSTGet Queued Pages
- POSTGet Banned Pages
- POSTGet Ignored Pages
Webᵀ Chat
Create Web Crawl
POST
/
crawl
/
create
Copy
curl --request POST \
--url https://api.webtranspose.com/v1/crawl/create \
--header 'Content-Type: application/json' \
--data '{
"url": "<string>",
"render_js": false,
"max_pages": 123,
"allowed_urls": [],
"banned_urls": []
}'Assistant
Responses are generated using AI and may contain mistakes.
Body
application/json
Response
200
Successful operation
Copy
curl --request POST \
--url https://api.webtranspose.com/v1/crawl/create \
--header 'Content-Type: application/json' \
--data '{
"url": "<string>",
"render_js": false,
"max_pages": 123,
"allowed_urls": [],
"banned_urls": []
}'