crawl4ai-mcp
3.5
The Crawl4ai MCP Server is designed to facilitate web crawling by leveraging crawl4ai, outputting content in Markdown format for Language Learning Models. It includes features for crawling web pages and handles common errors related to authentication, network connectivity, and web page access.
Top Comments
Crawl4ai MCP Server
An MCP server that offers web crawling capabilities using crawl4ai, providing Markdown output for Language Learning Models (LLM).
Features
- Crawl URLs and obtain Markdown content with citations.
Error Handling
- Ensure URLs are valid and accessible.
- Validate authentication token if used.
- Verify network connectivity.
- Service handles retries with different user agents if blocked by websites.
License
Licensed under the MIT License.