scrapeninja

ScrapeNinja is a high-performance web scraping MCP server designed to tackle common challenges encountered by developers when scraping various websites, including e-commerce and social networking platforms. This server leverages advanced features such as rotating proxies and smart retries to ensure reliable and efficient data extraction without the need for manual management in your applications.

Features

• Two Web Scraping Engines: /scrape for high-performance scraping with a real browser fingerprint and /scrape-js for full-fledged browser mode.
• Smart Retries & Rotating Proxies: Automatically handle retries and proxy rotation to increase success rates.
• Additional Tools: Includes cURL Converter, ScrapeNinja Playground, and Cheerio Playground for enhanced scraping capabilities.
• Improved Cloudflare Bypass: Enhanced ability to bypass Cloudflare and Akamai challenges.
• Expanded Proxy Pool: Default proxy pool expanded to improve success rates and accommodate increased demand.

MCP Tools

• cURL Converter: Easily convert cURL commands into Python or Node.js code.
• ScrapeNinja Playground: Test and develop scrapers directly in your browser.
• Cheerio Playground: Simplify web scraping tasks with a user-friendly interface.

Usage with Different Platforms

n8n

javascript
{
  "nodes": [
    {
      "parameters": {
        "httpMethod": "POST",
        "url": "https://api.scrapeninja.net/scrape",
        "jsonParameters": true,
        "options": {},
        "bodyParametersJson": "{\"url\": \"https://example.com\"}"
      },
      "name": "HTTP Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [450, 300]
    }
  ]
}

Make

{
  "module": "HTTP",
  "method": "POST",
  "url": "https://api.scrapeninja.net/scrape",
  "headers": {
    "Content-Type": "application/json"
  },
  "body": {
    "url": "https://example.com"
  }
}