Connect Claude, Claude Code, Cursor, or Codex to crawlie.app/mcp and let your agent run crawls and read reports.

What it is

Crawlie Cloud speaks the Model Context Protocol at:

https://crawlie.app/mcp

Point an MCP client at that URL and your agent can start crawls, poll them, and work with your reports — the same data you see in the dashboard, in your workspace, on your plan.

Connect your client

Pick your client for the exact setup:

In Claude (web or desktop), open Settings → Connectors → Add custom connector.
Paste the server URL:
```
https://crawlie.app/mcp
```
Claude opens a Crawlie sign-in — approve it and you're connected. No keys to paste.

Try: "Crawl example.com and walk me through the top fixes."

Custom connectors require a paid Claude plan.

Add the server with one command:

claude mcp add --transport http crawlie https://crawlie.app/mcp

Inside Claude Code, run /mcp and pick crawlie to sign in (OAuth, opens your browser).

Prefer a key over OAuth — e.g. for CI? Pass an API key instead:

claude mcp add --transport http crawlie https://crawlie.app/mcp \
  --header "Authorization: Bearer crw_YOUR_KEY"

Try: "Audit https://example.com and fix the top issues in this repo."

Add crawlie to ~/.cursor/mcp.json (all projects) or .cursor/mcp.json (this project):
```
{
  "mcpServers": {
    "crawlie": { "url": "https://crawlie.app/mcp" }
  }
}
```
Enable it under Settings → MCP — Cursor prompts a Crawlie sign-in the first time.

Or skip OAuth with an API key:

{
  "mcpServers": {
    "crawlie": {
      "url": "https://crawlie.app/mcp",
      "headers": { "Authorization": "Bearer crw_YOUR_KEY" }
    }
  }
}

Try (in Agent mode): "Run a crawl of example.com and open the templates that need fixing."

ChatGPT

Open Settings → Connectors (with developer mode enabled) and add a connector with the URL:
```
https://crawlie.app/mcp
```
Approve the Crawlie sign-in when ChatGPT opens it.

Codex CLI

Add the server to ~/.codex/config.toml:

[mcp_servers.crawlie]
url = "https://crawlie.app/mcp"

Codex signs you in on first use; or authenticate with an API key as a bearer token.

Try: "What's blocking example.com from being cited by AI search?"

Using something else? Any MCP client works: give it https://crawlie.app/mcp — clients that speak OAuth open a Crawlie sign-in, and everything else can send an API key as a bearer token.

The tools

Tool	What the agent gets
`crawl_site`	Start a crawl of a site (streams progress, or returns a job to poll).
`audit_url`	Audit a single URL.
`crawl_status`	Check on a running crawl.
`list_reports`	Browse the workspace’s reports.
`get_report`	Pull a full report.
`top_fixes`	The highest-impact fixes from a report, ranked.
`geo_gaps`	Where the site falls short for AI search.
`affected_urls`	Every URL a given issue affects.
`diff_reports`	Compare two crawls.

Good to know

Crawls started by an agent spend crawl credits like any other crawl, and respect your plan’s page caps.
The prompt patterns that work well (“audit this site and fix the top issues”) are collected in the prompt library.
Running fully local instead? The open-source crawler ships its own MCP server — same idea, your machine.

The hosted MCP server

What it is

Connect your client

The tools

Good to know