crawl4ai-mcp-server
84
Crawl4AI MCP Server is an intelligent server designed to enhance AI assistant systems by providing extensive search capabilities and web content understanding tailored for large-scale models. It supports multiple search engines and formats web content to be ideal for LLM processing.
Crawl4AI MCP Server
A smart information retrieval server based on MCP (Model Context Protocol) providing powerful search capabilities and web content understanding optimized for LLMs.
Features
- Multi-engine search capability supporting DuckDuckGo and Google.
- Optimized web content extraction for LLMs, filtering non-core content.
- Focused on information value, automatically identifies and retains key content.
- Multiple output formats supporting citation tracing.
- Built on high-performance asynchronous design with FastMCP.
Installation
- Ensure Python >= 3.9 and use a virtual environment.
- Clone the repository and install dependencies.
- Install Playwright browser.
Usage
- Provides tools like
searchandread_urlfor powerful web searches and content understanding.- Search with engines like DuckDuckGo and Google.
- Extract and format web content optimized for LLMs.
LLM Content Optimization
- Automatically identifies and retains main content.
- Filters noise like navigation bars and ads.
- Maintains URL citations for integrity.
Development Notes
Includes structure information with main implementations in index.py and search.py.
Configuration
Copy config_demo.json and configure Google API if needed.
License
MIT License.