POST
/
v1
/
web
curl --request POST \
  --url https://api.taam.cloud/v1/web \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "model": "scrape",
  "params": {}
}'
{
  "id": "<string>",
  "object": "<string>",
  "created": 123,
  "model": "<string>",
  "data": {},
  "usage": {
    "prompt_tokens": 123,
    "completion_tokens": 123,
    "total_tokens": 123
  },
  "system_fingerprint": "<string>"
}

The Website Crawling service allows you to systematically explore and extract content from websites.

​
Required Parameters

​
url
string
required

The base URL to start crawling from

​
Optional Parameters

​
Example Request

curl --request POST \
  --url https://crawl.taam.cloud/v1/web \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "url": "https://example.com",
    "maxDepth": 2,
    "includePaths": ["blog/*"],
    "excludePaths": ["admin/*"],
    "limit": 100,
    "scrapeOptions": {
      "formats": ["markdown"],
      "onlyMainContent": true
    }
  }'

​
Example Response

{
  "success": true,
  "id": "crawl_123abc",
  "url": "https://example.com"
}

​
Status Codes

Authorizations

​
Authorization
string
header
required

Enter your API key prefixed with 'Bearer '

Body

application/json

Response

200 - application/json

Successful response

The response is of type object.