Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.
AI Crawler Readiness
Can AI crawlers read this site? GPTBot/ClaudeBot verdicts, robots vs edge blocking, 47k-domain index
Install
$
claude mcp add --transport http ai-crawler-readiness https://shop.lumnika.com/ai-readiness/mcpInstall command taken from the project's README or registry entry. Always review code before running it.
Similar plugs
An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here…
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.
The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra