vibe-eyes

vibe-eyes

29
browser_automationdeveloper_tools

Vibe-Eyes is an MCP server designed to enhance interactions between large language models and browser-based games by providing real-time visual context and debug information. It captures and vectorizes canvas elements and exposes this data through the Model Context Protocol (MCP).

Vibe-Eyes

An MCP server that enables LLMs to "see" what's happening in browser-based games and applications through vectorized canvas visualization and debug information. Vibe-Eyes employs a client-server architecture, where a lightweight browser client captures canvas content and debug information. This data, sent to a Node.js server via WebSockets, is vectorized into compact SVG representations and made accessible to LLMs through the Model Context Protocol (MCP).

Key Capabilities

  • Captures and vectorizes canvas elements from browser games
  • Collects console logs and errors in real-time
  • Catches unhandled exceptions with full stack traces
  • Makes the visual and debug information available to LLMs via MCP
  • Creates a seamless debugging experience for developers working with LLMs

How It Works

  1. A lightweight client runs in the browser game/application
  2. The client captures canvas snapshots, console logs/errors, and unhandled exceptions
  3. Data is sent to the Vibe-Eyes server via WebSocket (avoiding CORS issues)
  4. The server vectorizes the canvas images and stores them with the debug information
  5. LLMs connect via the Model Context Protocol to access the latest data
  6. LLMs can "see" what's happening and help debug issues with full context

Components

1. Vibe-Eyes MCP Server (mcp.js)

  • Receives canvas snapshots via Socket.IO
  • Vectorizes images to compact SVG representation
  • Stores debug information
  • Exposes the data via Model Context Protocol (MCP)

2. Browser Client

  • Finds canvas elements in the page
  • Captures canvas content as data URLs
  • Intercepts console logs and errors

3. Vectorization Engine (vectorizer.js)

  • Converts raster images to vector SVGs
  • Optimizes SVGs for size and clarity

Getting Started

Installation

  • Clone the repository and install dependencies

Using with LLM Agents

  • Register the MCP server with your AI agent

Integrating with Games/Applications

  • Add the client to your browser application by including the required scripts

Using with Claude or Other LLMs

  • The MCP server exposes a tool for LLMs to access the latest visual and debug information