Quickstart

Download Python SDK

Download the Python SDK.

pip install webtranspose

Not a Python user? Use Web Transpose via API.

Get your API Key

Get your API key from the Web Transpose Dashboard.

os.environ['WEBTRANSPOSE_API_KEY'] = "YOUR API KEY"

Crawl a Website

Your crawls can be viewed and monitored in the Web Transpose Crawls Dashboard.

Crawl a Website (3 lines of code)

import webtranspose as webt

crawl = webt.Crawl(
"https://www.example.com",
max_pages=100,
render_js=True,
api_key="YOUR_WEBTRANSPOSE_API_KEY", # optional, if you want to run on cloud
)
await crawl.crawl() # crawl.queue_crawl() for async

Download the Crawled Data (1 line of code)

Access the crawled data (1 line of code)

Scrape a Website

Your scrapers can be viewed and monitored in the Web Transpose Scraper Dashboard.

Build a Scraper (1 line of code)

import webtranspose as webt

schema = {
"Merchant Name": "string",
"Title of Product": "string",
"Product Photo URL": "string",
}

scraper = webt.Scraper(
schema, 
render_js=True, 
api_key="YOUR_WEBTRANSPOSE_API_KEY", # optional, if you want to run on cloud
)

Scrape a website (1 line of code)

SERP API

Run the SERP API doing the following:

Regular web search.

Web search + AI filtering

Get Started

Webᵀ Crawl

Webᵀ Scrape

Query Website as Vector DB

AI Web Search

Download Python SDK

Get your API Key

Crawl a Website

Scrape a Website

SERP API

Get Started

Webᵀ Crawl

Webᵀ Scrape

Query Website as Vector DB

AI Web Search

​Download Python SDK

​Get your API Key

​Crawl a Website

​Scrape a Website

​SERP API

Download Python SDK

Get your API Key

Crawl a Website

Scrape a Website

SERP API