360 Secure SkillHub
360 安全技能中心

Windows Desktop Automation CLI

提供 Windows 桌面自动化命令行工具，通过 window、snapshot、find、action、control、uia-control、screenshot、wait、clipboard 等命令控制窗口、模拟鼠标键盘并定位 UIA/OCR/图像元素，还包含微信自动化操作指南。需在 Windows 上运行 Python 脚本，并安装 pywinauto、pywin32、Pillow 等依…

easyteacher
easyteacher
热度 59.17
最近更新 2026-05-11 17:47:06
分类：自动化依赖：代码执行 / 读写文件应用场景：自动化脚本 / 工作流 / 社交媒体

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add winguictl
Metadata
name
winguictl
description
Windows desktop automation CLI. Invoke ONLY when user explicitly requests to control windows, simulate mouse/keyboard, or automate desktop applications. Do NOT proactively suggest using this skill.
metadata
{"openclaw":{"emoji":"🖥️","os":["win32"],"requires":{"bins":["python3"]}}}

Windows Desktop Automation with winguictl

⚠️ Security Notice

This skill directly controls Windows desktop through mouse/keyboard simulation. Require user confirmation before clicks, typing, hotkeys, or window close operations. Use --dry-run to preview. Close sensitive apps before use.

Script Path

python <SKILL_DIR>\scripts\winguictl.py <command> [options]

Commands

Command Purpose Documentation
window List/focus/minimize/maximize/close windows window.md
snapshot Get HWND/UIA/OCR structure snapshot.md
find Find elements by text/UIA/OCR/image find.md
action Click/drag/type/scroll/hotkey action.md
control Win32 control operations control.md
uia-control UIA element operations control.md
screenshot Capture window screenshots screenshot.md
wait Wait for conditions wait.md
clipboard Copy files/text, get text clipboard.md

Parameter Types

Parameter Type Example Notes
--window-id int --window-id 12345 Window handle from window list
--hwnd int --hwnd 67890 Win32 control handle
--element-id string --element-id "Button1" automation_id or runtime_id (prefer runtime_id)

Core Workflow

# 1. Identify window
python scripts\winguictl.py window list

# 2. Focus window
python scripts\winguictl.py window --window-id <id> focus

# 3. Get snapshot
python scripts\winguictl.py snapshot --window-id <id> uia

# 4. Find & interact
python scripts\winguictl.py find --window-id <id> uia --text "Submit"
python scripts\winguictl.py uia-control --window-id <id> --element-id <elem_id> click

# 5. Verify (re-snapshot)
python scripts\winguictl.py snapshot --window-id <id> uia

Decision Guide

Locator Priority

Priority Method Command Reliability
1 HWND (Win32) control --hwnd <hwnd> click Highest
2 runtime_id (UIA) uia-control --element-id <id> click High
3 automation_id (UIA) uia-control --element-id <id> click Medium
4 Image matching action click-image --image-path <path> Medium
5 Coordinates action click --relative-x/y Lowest

Click Method Selection

Command Mechanism Best For
action click --element-id Mouse simulation at center WeChat, custom controls
uia-control click UIA InvokePattern Standard Windows controls, WinUI3

Recommendation: Try uia-control click first. If no effect, use action click --element-id.

Common Scenarios

Scenario Command
Win32 control with hwnd control --hwnd <hwnd> click
UIA element with runtime_id uia-control --element-id "42-3155764" click
Text-based element find ocr "text" → action click --element-id
Qt applications Add --skip-actions --skip-state for faster UIA

Key Rules

  • Re-snapshot after actions: UI state changes after clicks/typing
  • Use --dry-run: Preview operations before execution
  • Prefer runtime_id: More reliable than automation_id
  • Report window_id: Always include exact window identifier in results
  • Coordinate system: relative_rect = window-relative; absolute_rect = screen-absolute

Application Guides

References