Crawlseo
Open-source SEO monitoring. GSC + Site Crawler + Core Web Vitals in one dashboard.
Install
npx prisma migrate dev --name initclaude mcp add crawlseo -- npx prisma migrate dev --name init{
"mcpServers": {
"crawlseo": {
"command": "npx",
"args": [
"prisma",
"migrate",
"dev",
"--name",
"init"
]
}
}
}Install command taken from the project's README or registry entry. Always review code before running it.
README excerpt
Read full README on GitHub →Open-source SEO monitoring for founders, not SEO specialists
Google Search Console, a site crawler, Core Web Vitals and an MCP server in one self-hosted dashboard. Free forever.
Why crawlseo?
| crawlseo | OpenSEO | Ahrefs | Semrush | Moz | |
|---|---|---|---|---|---|
| Price | Free | $10/mo | €119/mo | $139/mo | $49/mo |
| Self-hosted | ✅ | ✅ | ❌ | ❌ | ❌ |
| GSC integration | ✅ | ✅ | ✅ | ✅ | ✅ |
| Site crawler | ✅ (2000 pages) | ✅ | ✅ | ✅ | ✅ |
| Core Web Vitals | ✅ | ❌ | ❌ | ✅ | ❌ |
| MCP Server | ✅ (10 tools) | ✅ (24 tools) | ❌ | ❌ | ❌ |
| AI agent ready | ✅ | ✅ | ❌ | ❌ | ❌ |
| Keyword Research | ✅ (BYOK) | ✅ | ✅ | ✅ | ✅ |
| Backlinks | ✅ (BYOK) | ✅ | ✅ | ✅ | ✅ |
| Open source | ✅ MIT | ✅ | ❌ | ❌ | ❌ |
| Your data stays yours | ✅ | ✅ | ❌ | ❌ | ❌ |
BYOK = Bring Your Own Key. Keyword research and backlink data use DataForSEO (optional). Google Autocomplete suggestions work as a free fallback.
Features
🔍 GSC Analytics
Keywords, pages, clicks, impressions, position tracking with 28-day comparison and delta indicators.
Top keywords with position badges, clicks, impressions, and CTR
🕷️ Site Crawler
Crawl up to 2,000 pages with concurrent fetching. Health score, 16 issue types, content scoring, and remediation guidance.
The crawler (CrawlSEOBot/1.0 (+https://crawlseo.cloud)) follows robots.txt per RFC 9309: each host (the bare domain and www included) is checked against its own robots.txt, redirect targets are checked before they are followed, a robots.txt that answers 5xx or times out skips that host, and Crawl-delay is honoured. The crawl page lists what robots.txt kept it from.
Crawl results with health score, issue breakdown, and per-page audit data
🤖 MCP server: AI agent integration
10 tools for Claude Code, Claude Desktop, and Cursor. Query your SEO data, run crawls, and find opportunities without leaving the terminal.
MCP setup page with connection config, setup guides, and available tools
More features
| Feature | Description | |
|---|---|---|
| ⚡ | Core Web Vitals | LCP, CLS, INP, TTFB via PageSpeed Insights with mobile/desktop comparison |
| 🔑 | Keyword Research | DataForSEO-powered keyword ideas with volume, difficulty, CPC. Free Google Autocomplete fallback |
| 🔗 | Backlinks | Backlink profile, referring domains, anchor text, dofollow/nofollow analysis |
| 📊 | Rank Tracking | Historical position snapshots with saved keywords and notes |
| 💡 | SEO Opportunities | Striking distance keywords, low CTR, content decay, cannibalization detection |
| 🔔 | Alerts | Traffic drops, position changes, new 404s, vitals degradation, by email, Slack, Telegram or webhook |
| 📥 | CSV Export | Export keywords and pages data for offline analysis |
Quick Start
git clone https://github.com/crawlseo/crawlseo.git
cd crawlseo
cp .env.example .env.local
# Add your Google OAuth credentials to .env.local
docker compose up -d db
npm install
npx prisma migrate dev --name init
npm run dev
Open http://localhost:3000, sign in with Google, and add your first site.
🔑 Getting Google OAuth credentials
Similar plugs
Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here…
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.
The fastest path to AI-powered full stack observability, even for lean teams.