REST API overview

Basics

Base URL:   https://crawlie.app/v1
Auth:       Authorization: Bearer crw_...      (an API key)
Workspace:  x-crawlie-team: <workspace-id>     (which workspace to act in)

Everything the dashboard does goes through this API, so anything you can click, you can script. Keys come from Settings → API keys.

What’s on the surface

Area	What you can do
Crawls	Start site crawls or URL audits, list running crawls, cancel.
Reports	List, fetch full reports, page index, share/unshare, delete.
Diff	Compare two reports.
Projects	Manage projects, schedules, and alert settings.
Rules	Manage custom rule packs, preview against the last crawl.
Search	Build and query a project’s search index.
Team	Members, invites, workspace settings.

A taste

Start a crawl and fetch the report when it’s done:

# start
curl -X POST https://crawlie.app/v1/crawls \
  -H "Authorization: Bearer crw_..." \
  -H "x-crawlie-team: <workspace-id>" \
  -H "content-type: application/json" \
  -d '{"url": "https://example.com"}'

# later: grab the report
curl https://crawlie.app/v1/reports/<report-id> \
  -H "Authorization: Bearer crw_..." \
  -H "x-crawlie-team: <workspace-id>"

Limits

API-started crawls follow the same rules as the dashboard: your plan’s page caps apply and each crawled page spends a credit. If a crawl would exceed the workspace’s remaining credits, the API returns a 402 rather than charging you.

For agent use, the MCP server is usually the nicer surface — same capabilities, self-describing tools.