crawl4ai-mcp

crawl4ai-mcp

15
browser_automation

The Crawl4AI MCP Server is a Model Context Protocol server designed to integrate web scraping and crawling capabilities within Cursor AI. It allows tools to scrape single pages and crawl entire websites, making it useful within the Cursor Composer's agent mode framework.

Crawl4AI MCP Server

A Model Context Protocol (MCP) server implementation that integrates Crawl4AI with Cursor AI, providing web scraping and crawling capabilities as tools for LLMs in Cursor Composer's agent mode.

System Requirements

  • Python 3.10 or higher installed.

Current Features

  • Single page scraping
  • Website crawling

Installation

Basic setup instructions also available in the Official Docs for MCP Server QuickStart.

Tools Provided

  1. scrape_webpage

    • Description: Scrapes the content and metadata from a single webpage using Crawl4AI.
    • Parameters: url (string, required): The URL of the webpage to scrape.

  2. crawl_website

    • Description: Crawls a website starting from the given URL up to a specified depth and page limit using Crawl4AI.
    • Parameters: url (string, required), crawl_depth (integer, optional), max_pages (integer, optional).