Skip to content
NewsProduct launch

GitHub Copilot Can Now Click, Type, And Drag Inside Desktop Apps

· by Pondero Newsdesk

The short version

GitHub shipped a public-preview computer-use mode for Copilot CLI and the Copilot desktop app on October 1, 2026, letting the agent operate GUI-only software that has no API, CLI, or MCP integration.

GitHub Copilot Can Now Click, Type, And Drag Inside Desktop Apps

GitHub shipped a public-preview "computer use" mode on October 1 that lets Copilot read a desktop app's screen, click its controls, type into its fields, and drag items between windows, per GitHub's changelog. The target is software that has never offered an API in the first place.

What shipped

The feature is live in public preview inside Copilot CLI and the Copilot desktop app on macOS and Windows, according to the changelog. Copilot can read an app's accessible content and visual context, click controls, enter and edit text, press keys, scroll, drag, and move through multi-step workflows across more than one application at a time. GitHub's own launch demo shows Copilot stepping through an expense-report workflow inside Safari. The company frames the feature as a fallback for legacy and GUI-only tools that provide no API, command-line interface, or MCP server, not as a general-purpose replacement for those integrations.

Turning it on takes one command: /computer on in Copilot CLI, or the Computer Use toggle under Settings in the desktop app, per GitHub's documentation. Enabling it activates a bundled plugin that runs its own MCP server locally, reading the screen through the operating system's accessibility tree and falling back to screenshots when it needs visual context, The New Stack reported. On macOS, Copilot also walks the user through granting Accessibility and Screen Recording permissions the first time it needs them.

Every action still runs through an approval gate. Developers can check their current permission mode with /permissions show and choose to allow an app for one session, mark it "Always allow," or decline outright, with deny rules overriding both, according to The New Stack. An approval saved in the CLI carries over to the desktop app on the same machine, and removing an app from the always-allowed list only blocks future sessions, it does not revoke access already granted inside a session that is still running. Stopping an in-progress action requires a separate step: pressing Esc twice in the CLI, or clicking Stop (or pressing Esc) in the desktop app.

Enterprise admins get an override layer on top of that. A managed-settings.json policy can disable computer use org-wide or restrict whether individual developers can bypass its approval prompts, and that restriction applies across the Copilot app, CLI, and VS Code alike, per The New Stack's reporting on GitHub's documentation. GitHub's broader default-enablement policy for Business and Enterprise plans, which starts auto-enabling unconfigured features on October 22, explicitly excludes preview features, so computer use will not turn itself on for enterprise seats by default.

A preview feature can now click through screens no API ever covered

Computer use changes what counts as automatable. Until now, an agent that needed to touch a 20-year-old desktop app with no API, no CLI, and no MCP server had no path in. GitHub is explicitly positioning this as the last resort, not the first choice: its own guidance recommends reaching for a direct API, MCP server, terminal command, filesystem tool, or dedicated browser tool first, because those give an agent more structured, predictable information than clicking around a screen does. GitHub itself warns that the feature can misfire. Its documentation, as relayed by The New Stack, flags that a change in window timing or state can make Copilot repeat an action or stall, that the agent may click the wrong control or type into the wrong field on dynamic or complex interfaces, and that unexpected on-screen content combined with an ambiguous instruction can lead to actions affecting a user's device, data, or connected accounts. Sensitive information visible in a window, GitHub notes, can also become part of what the agent reads as context. For an operator deciding whether to turn this on for a shared machine, that permission model (what carries over, what a revoked approval actually blocks, and what only Esc actually stops) is worth reading closely before the first "Always allow" click.

Context and reactions

GitHub is not first to ship this. OpenAI added computer use to Codex in April, and Anthropic brought broader computer-use capability to Claude Code and Claude Cowork on macOS earlier in 2026, according to The New Stack's reporting, framing GitHub's move as catching up to rivals rather than opening new ground. The strategic debate over how far this pattern should go is already public: OpenAI president Greg Brockman argued last month that agents using the same interfaces humans do could save the industry from building and maintaining a separate connector for every piece of software, a broader bet than GitHub's own "try the API first" framing suggests it is willing to make yet.

What to watch next

Watch whether GitHub moves computer use from public preview toward general availability, and whether the October 22 default-enablement policy for Business and Enterprise plans is extended to cover it once the preview tag comes off. The first independent security writeups on an agent that can click through arbitrary desktop UI, rather than operate inside a sandboxed API surface, are the other thing worth tracking as adoption grows beyond early testers.

Sources