Prompt library

Drive Crawlie from your AI agent

Battle-tested prompts for the Crawlie MCP server. Connect it once, then pick a prompt and swap in your URL.

Diagnose — read-only. Works in any client (Claude Desktop, web, Cursor). It crawls and reports.

Diagnose + fix — agentic, end-to-end. In Claude Code it edits your repo and re-crawls to verify; a chat client hands the changes back.

Connect your agent

Install the Crawlie MCP once. Run npm i -g crawlie first to put crawlie-mcp on your PATH, then add it to your client. Full setup →

Claude Code

Run in your terminal

claude mcp add crawlie crawlie-mcp

Claude Desktop

Add to claude_desktop_config.json

{
  "mcpServers": {
    "crawlie": { "command": "crawlie-mcp" }
  }
}

Cursor

Add to ~/.cursor/mcp.json

{
  "mcpServers": {
    "crawlie": { "command": "crawlie-mcp" }
  }
}

Windsurf

Add to ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "crawlie": { "command": "crawlie-mcp" }
  }
}

Cline

Add via MCP Servers → Configure

{
  "mcpServers": {
    "crawlie": { "command": "crawlie-mcp" }
  }
}

VS Code

Add to .vscode/mcp.json

{
  "servers": {
    "crawlie": { "command": "crawlie-mcp" }
  }
}

Audit & fix

Crawl a site, surface what is broken, and turn findings into concrete changes.

Full site audit

Diagnose

A prioritised overview of a whole site in one shot.

Use the crawlie MCP to crawl https://example.com. Give me the one-line headline, the Health and GEO scores, and the top 5 fixes in priority order. Keep it tight.

crawl_sitetop_fixes

Audit a single page

Diagnose

Deep-check one important URL without crawling the whole site.

Audit https://example.com/pricing with the crawlie MCP. List every issue grouped by severity, and for each one tell me exactly how to fix it.

audit_url

Audit, fix, verify, PR

Diagnose + fix

Close the whole loop in your repo with Claude Code.

Crawl https://example.com with the crawlie MCP, fix the top 10 issues directly in this repo, then re-crawl and diff to confirm the Health and GEO scores went up. Open a PR summarising what changed and the score delta.

crawl_sitetop_fixesdiff_reports

Clear every error-level issue

Diagnose + fix

Hunt down and resolve the must-fix problems in code.

Crawl https://example.com with the crawlie MCP. For every error-level issue, find the affected pages in this codebase, apply the fix, and re-crawl to confirm zero error-level issues remain.

crawl_siteaffected_urlsexplain_issue

GEO · AI-search readiness

Make pages answerable and citable by ChatGPT, Perplexity, and Google AI Overviews.

GEO gap analysis

Diagnose

See which AI-readiness signals your pages are missing, at scale.

Crawl https://example.com with the crawlie MCP and break down the GEO gaps: how many indexable pages are missing authorship, dates, structured data, semantic HTML, answer-readiness, and question headings. Then tell me what to fix first for the biggest GEO score gain.

crawl_sitegeo_gaps

Top GEO fixes only

Diagnose

Skip the SEO noise and rank just the GEO work.

Using the crawlie MCP on the latest report, give me only the GEO fixes (category = geo), ranked by impact, with a one-line action for each.

top_fixesgeo_gaps

Make a page citable by AI

Diagnose + fix

Rewrite one page in code so answer engines will quote it.

Audit https://example.com/blog/my-post with the crawlie MCP, focused on GEO. Edit the page source in this repo so ChatGPT, Perplexity, and Google AI Overviews can understand and cite it: fix the headings, add the missing structured data and byline, then re-audit to confirm the GEO issues cleared.

audit_urlexplain_issue

Site-wide GEO sweep

Diagnose + fix

Add structured data and authorship everywhere it is missing.

Use the crawlie MCP to find every page missing structured data or authorship. Add the correct JSON-LD and bylines in the source for each, then diff a fresh crawl to prove the GEO gaps closed.

crawl_sitegeo_gapsdiff_reports

Monitoring & trends

Track scores over time and prove that your fixes actually landed.

Did my fixes land?

Diagnose

Diff a fresh crawl against the previous one.

Re-crawl https://example.com with the crawlie MCP and diff it against the previous report. Did the issues I fixed actually resolve? Call out any new regressions and any score changes.

crawl_sitediff_reports

Weekly health check

Diagnose

A repeatable status read you can run on a schedule.

Crawl https://example.com with the crawlie MCP, compare it to last week’s report, and summarise: current Health and GEO scores, what changed, and the single most important thing to fix this week.

crawl_sitelist_reports

Catch and fix a regression

Diagnose + fix

Find what newly broke and repair it in code.

Re-crawl https://example.com with the crawlie MCP and diff against the last report. For any newly-introduced issue, fix it in this repo and re-crawl to confirm the regression is gone.

diff_reportsaffected_urls

Custom extraction

Pull structured data off every page while you crawl.

Scrape a field site-wide

Diagnose

Use a CSS or regex extractor across the crawl.

Crawl https://example.com with the crawlie MCP and add a custom extractor that pulls the product price from the CSS selector .price on every page. Return a table of URL and price, sorted high to low.

crawl_site

Audit canonical tags

Diagnose

Extract and validate an attribute across the site.

Crawl https://example.com with the crawlie MCP, extracting the href attribute of link[rel="canonical"] on each page. Flag any pages whose canonical does not match their own URL.

crawl_site

Learn the checks

Understand any rule and what crawlie looks for, in plain English.

Explain a rule

Diagnose

Why a finding matters and how to resolve it.

Using the crawlie MCP, explain the rule "geo-not-answerable": why it matters for AI search, how to fix it, and what happens if I ignore it.

explain_issue

List everything crawlie checks

Diagnose

The full rule catalogue with severity and category.

Ask the crawlie MCP to list every audit rule it checks, with category and severity, then group them by category so I can see the coverage.

list_rules

Prompts use https://example.com as a placeholder — swap in your own site. New to the MCP? Start with the MCP docs or the Claude Code plugin for one-step setup.