mcp-playwright-scraper

mcp-playwright-scraper

1
browser_automationfile_systems

MCP Playwright Scraper is a server designed to scrape web content and convert it into Markdown format using tools like Playwright, BeautifulSoup, and Pypandoc. It aims to simplify the process of extracting and transforming web data into a more portable and readable format.

Overview

This MCP server provides a simple tool for scraping web content and converting it to Markdown format. It uses:

  • Playwright: For headless browser automation to handle modern web pages including JavaScript-heavy sites
  • BeautifulSoup: For HTML parsing and cleanup
  • Pypandoc: For high-quality HTML to Markdown conversion

Tools

The server implements a single tool:

  • scrape_to_markdown: Scrapes content from a URL and converts it to Markdown
    • Required parameter: url (string) - The URL to scrape
    • Optional parameter: verify_ssl (boolean) - Whether to verify SSL certificates (default: true)

Installation

Using uv (recommended)

When using uv no specific installation is needed. We will use uvx to directly run mcp-playwright-scraper.

Using PIP

Alternatively you can install mcp-playwright-scraper via pip: After installation, you can run it as a script using:

Prerequisites

  • Python 3.11 or higher
  • Playwright browser dependencies
  • Pandoc (optional, will be automatically installed by pypandoc if possible)

Configuration

Usage with Claude Desktop

Usage with Claude Code

Usage with Zed

Usage with Cursor

Usage

Once configured in Claude Desktop, you can explicitly use the scraper with a prompt like:

Debugging

You can use the MCP inspector to debug the server:

Development

Building and Publishing

To prepare the package for distribution:

  1. Sync dependencies and update lockfile:
  2. Build package distributions:
  3. Publish to PyPI:

License

This MCP server is licensed under the Apache License, Version 2.0. You are free to use, modify, and distribute the software, subject to the terms and conditions of the Apache License 2.0. For more details, please see the LICENSE file in the project repository or visit http://www.apache.org/licenses/LICENSE-2.0.