MSPaint-MCP-Server

MSPaint-MCP-Server

0
os_automation

This project automates the interaction with MSPaint using an AI agent and the Model Context Protocol. It uses tools like `pywinauto` and Google Gemini model to allow natural language-driven operations on the Paint application, showcasing the integration of AI with legacy software.

Model Context Protocol (MCP) MSPaint App Automation

  • Demonstrates automation of MSPaint using an AI agent with MCP.
  • Leverages pywinauto for controlling Paint and fastmcp to define callable tools by an AI agent.
  • The AI agent uses Google’s Gemini model for tasks like drawing rectangles and adding text via natural language.

Introduction

  • Showcases MSPaint automation by AI through natural language instructions via MCP, defining and calling specific functions in Python.

Project Structure

  • Contains files such as mcp_server.py for Paint automation and mcp_client.py for interaction between AI and server.

Requirements

  • Python 3.11+, Conda, Google Gemini API key, pywin32, pywinauto, fastmcp, python-dotenv, google-genai.

Setup

  • Create a Conda environment, install dependencies, set up the Gemini API key in the .env file.

Usage

  • Run the MCP client to start automation through the AI agent's interaction with the MCP server.

How It Works

  • MCP Server defines tools for Paint manipulation and exposes them for AI agent use.
  • MCP Client interacts with the server using Google Gemini model to generate instructions and handle responses.

Troubleshooting

  • Common issues include permission, coordinate, tool selection, and API key set up troubles.