Desktop Control
Advanced desktop automation with pixel-perfect mouse control, keyboard input, screen capture, window management, and clipboard operations. Built on PyAutoGUI.
# Desktop Control Advanced desktop automation with pixel-perfect mouse control, keyboard input, screen capture, window management, and clipboard operations. Built on PyAutoGUI. Desktop Control provides comprehensive desktop automation capabilities, allowing OpenClaw to interact with any application on your computer. It offers pixel-perfect mouse control, fast keyboard input, screen capture with image recognition, window management, and clipboard operations — all with built-in safety features. ## How It Works The skill uses PyAutoGUI as its core automation engine, with Pillow for image processing, OpenCV for image recognition, and PyGetWindow for window management. It provides a Python API through the DesktopController class that handles mouse movements (including smooth bezier curves), keyboard events, screen captures, and window state management. Safety mechanisms include failsafe corners, pause control, and an optional approval mode. ## Key Features - **Mouse Control**: Absolute and relative positioning, smooth movement, left/right/middle/double click, drag-and-drop, and scrolling - **Keyboard Input**: Fast text typing with configurable WPM, hotkey shortcuts, special keys, key combinations, and hold/release control - **Screen Operations**: Full-screen and region screenshots, image recognition via OpenCV, pixel color detection, and multi-monitor support - **Window Management**: List all windows, activate by title, get window info, and minimize/maximize control - **Clipboard Operations**: Copy text to and read text from the system clipboard - **Safety Features**: Failsafe corner abort, emergency pause, optional approval mode for actions, bounds checking, and action logging ## Requirements - **Python Packages**: PyAutoGUI, Pillow, and PyGetWindow are required. OpenCV (opencv-python) is optional for image recognition - **Operating System**: Works on macOS, Linux, and Windows. Some features may require elevated privileges - **Display**: A graphical desktop environment is required (not compatible with headless servers) ## Use Cases - **Form Filling**: Automatically fill out web forms or desktop application fields with typed input - **Application Automation**: Control any desktop application through mouse and keyboard simulation - **Screen Monitoring**: Capture screenshots, detect specific UI elements via image recognition, and respond to visual changes - **File Management**: Drag and drop files between folders, multi-select with Ctrl+click - **Testing & QA**: Automate manual GUI testing workflows across desktop applications ## Installation Install via: `npx clawhub@latest install desktop-control`