Desktop Automation MCP Server MCP Server

About

## Desktop Automation MCP Server: System-Level Control The **Desktop Automation MCP Server** enables Google Antigravity to control desktop applications, simulate user input, and automate GUI interactions across Windows, macOS, and Linux systems. ### Why Desktop Automation MCP? - **GUI automation** - Control any desktop application - **Cross-platform** - Works on Windows, macOS, and Linux - **Input simulation** - Keyboard and mouse control - **Screen interaction** - Find and click UI elements - **Window management** - Control application windows ### Key Features #### 1. Mouse and Keyboard Control ```python # Move mouse and click await mcp.call("desktop", "mouse_move", { "x": 500, "y": 300 }) await mcp.call("desktop", "mouse_click", { "button": "left", "clicks": 2 }) # Type text await mcp.call("desktop", "type_text", { "text": "Hello World", "interval": 0.05 }) # Press keyboard shortcut await mcp.call("desktop", "hotkey", { "keys": ["ctrl", "s"] }) ``` #### 2. Screen Element Location ```python # Find element by image element = await mcp.call("desktop", "locate_image", { "image_path": "/templates/button.png", "confidence": 0.9 }) if element: await mcp.call("desktop", "click_element", { "x": element["x"], "y": element["y"] }) # Find by text (OCR) text_loc = await mcp.call("desktop", "locate_text", { "text": "Submit", "region": {"x": 0, "y": 0, "width": 1920, "height": 1080} }) ``` #### 3. Window Management ```python # Get active window window = await mcp.call("desktop", "get_active_window", {}) print(f"Active: {window[\"title\"]}") # Focus specific window await mcp.call("desktop", "focus_window", { "title": "Visual Studio Code" }) # Resize window await mcp.call("desktop", "resize_window", { "title": "Terminal", "width": 800, "height": 600 }) ``` #### 4. Screen Capture ```python # Capture screen region screenshot = await mcp.call("desktop", "screenshot", { "region": {"x": 0, "y": 0, "width": 500, "height": 400}, "format": "png" }) # Record screen await mcp.call("desktop", "start_recording", { "output": "/recordings/demo.mp4", "fps": 30 }) ``` ### Configuration ```json { "mcpServers": { "desktop": { "command": "npx", "args": ["-y", "@anthropic/mcp-desktop-automation"], "env": { "DISPLAY": ":0" } } } } ``` ### Use Cases **GUI Testing**: Automate UI testing for desktop applications. **Data Entry**: Automate repetitive data entry tasks. **Legacy Integration**: Interact with apps lacking APIs. **Workflow Recording**: Record and replay user workflows. The Desktop Automation MCP Server enables system-level automation.

Installation

{
  "mcpServers": {
    "desktop-automation": {
      "mcpServers": {
        "desktop-automation": {
          "args": [
            "-y",
            "mcp-desktop-automation"
          ],
          "command": "npx"
        }
      }
    }
  }
}

How to Use

1. 1Uses RobotJS for control
2. 2Mouse and keyboard automation
3. 3Screenshot capabilities