GitHub Copilot Can Now Drive Your Desktop Apps: Three Safe Ways to Try It

For years, the joke about software automation was that every workflow worth automating already had an API. Everything else was stuck in "human-required" territory: clicking through a legacy GUI, copying numbers between two apps that refuse to talk, or walking a spreadsheet through a vendor portal that predates REST.

That line just moved. GitHub Copilot's new computer use feature is now in public preview in the GitHub Copilot app and Copilot CLI on both macOS and Windows. Copilot can now see your desktop, read accessible UI content, click controls, enter and edit text, press keys, scroll, drag, and navigate workflows across applications — including legacy and GUI-only software that has no API, command line, or MCP server.

This is not about replacing the keyboard and mouse. It is about letting a capable agent operate the parts of your workday that still live outside the API era, while keeping you in the loop and in control.

What Computer Use Actually Means

Computer use in Copilot is an accessibility-first approach to desktop automation. Copilot uses the same accessibility and visual context tools that screen readers and assistive technologies rely on to understand what is on screen. It can read UI labels, see form fields, identify buttons, and observe state changes. Then it can perform the same actions a person would: click, type, tab, scroll, drag, and switch between apps.

The important detail is that it works on the surface of the operating system. It does not require the target application to expose an API. That opens up a huge category of tools that were previously off-limits to automation: old ERP clients, internal portals, desktop-only design tools, proprietary dashboards, and applications whose vendors never got around to building a webhook.

There are two ways to turn it on:

• Copilot CLI: run /computer on. Use /computer show to check status and /computer off to disable.
• GitHub Copilot app: open Settings, select Computer Use, and enable it.

On macOS, Copilot walks you through the required Accessibility and Screen Recording permissions. On Windows, it integrates with the same accessibility framework. Organization-managed settings can disable the feature centrally, which is exactly what you want for enterprise rollouts.

You Stay in Control

Before anyone pictures an agent rearranging their inbox unsupervised, GitHub built in approval controls. Copilot asks for permission before controlling an app. You can review the request, approve it, and always reset which apps are allowed to be controlled automatically. For teams, admins can disable computer use entirely through organization policy.

This approval model is the right default. Desktop automation is more variable than API automation. A web endpoint has a contract; a legacy Windows app has quirks. Requiring approval keeps the human in the loop until the workflow has proven itself reliable.

Three Practical Workflows to Try First

The best way to evaluate computer use is to pick a repetitive task that is annoying, low-risk, and clearly bounded. Here are three starting points that fit well.

1. Consolidate Notifications and Status Updates

Many professionals start the day by checking four or five different places: email, a ticketing system, a project dashboard, a chat tool, and a monitoring app. None of these individually is hard to open, but the context switching adds up.

With computer use, you can ask Copilot: "Check the three open tickets in our desktop support tool, summarize the newest email thread about the release, and give me a single status update I can paste into standup." Copilot can navigate each app, read the relevant sections, and hand you a consolidated summary. You approve each app interaction, review the result, and paste where you need it.

This is a safe first workflow because it is read-heavy and produces a draft for your review. It does not change anything in the source systems.

2. Move Data Through a GUI-Only Process

Every company has that one process that only works through a desktop client or web portal with no stable API. It might be submitting an expense report, updating a supplier record, booking a resource, or uploading a file to a compliance system. The steps are always the same: open the app, navigate five menus, paste values from a spreadsheet, click submit, and save the confirmation.

Computer use lets you describe the outcome and the source data, then watch Copilot execute the clicks. For example: "Using the values in this Excel sheet, create the three new vendor records in the procurement portal and save the confirmation numbers back to column D." You approve each app, Copilot performs the steps, and you verify the confirmations before closing the tab.

Start with one record, not thirty. Once you see it succeed consistently, you can scale up.

3. Cross-Application Content Drafting

Modern work often involves moving content between apps that do not share a clean integration. A marketing manager might need to pull campaign numbers from a desktop analytics tool, summarize them, and paste the result into a presentation. A developer might need to reproduce a bug reported in a desktop crash reporter, capture the details, and open a GitHub issue.

With computer use, you can say: "Pull the top five metrics from this dashboard, summarize the week-over-week change, and draft a two-bullet update I can add to the slide deck." Copilot reads the dashboard, extracts the values, and gives you a draft. You edit it, approve the paste, and move on.

Where It Connects to the Bigger Microsoft Ecosystem

Computer use in GitHub Copilot fits neatly into the broader Microsoft AI strategy. The new Copilot is organized around Home, Code, and Autopilot, with the goal of giving users one place to ask, build, delegate, and automate. The GitHub Copilot app and CLI are the developer-focused surfaces of that same family.

For Windows developers, this is especially relevant. Windows remains the dominant enterprise desktop, and it is where most legacy business applications live. Bringing agentic control to those applications without requiring each vendor to build an integration is a meaningful unlock. It also complements the work happening in Microsoft Foundry and the Microsoft Agent Framework: you can build API-native agents in Foundry for the systems that support it, and use Copilot computer use for the desktop surface that does not.

How to Get the Best Results

Computer use works best when you describe the outcome, the apps involved, and any constraints. A vague prompt like "do my expense report" will struggle. A specific prompt like "Open the expense portal, create a new report for my trip last week, fill the dates and amounts from the receipts folder, and stop before submitting so I can review" will do much better.

A few tips from the early preview:

• Start read-only. Ask Copilot to gather or summarize information before you ask it to click Submit.
• Use exact app names. "Open Contoso Invoicer" is clearer than "the billing app."
• Set guardrails. Tell Copilot to stop at review steps, avoid certain buttons, or ask before sending email.
• Run one workflow at a time. Parallel desktop automation is harder to supervise. Serial workflows are easier to verify.
• Keep the target apps simple. Clean, responsive UIs with clear labels work better than heavily customized, overloaded dashboards.

What to Watch Next

Public preview means this is the right time to experiment, not the right time to hand over your most critical workflow unsupervised. The most valuable thing you can do this week is identify one tedious desktop task that happens repeatedly, try it with Copilot, and document what works.

The companies that will benefit most are the ones that treat computer use as a bridge, not a replacement for proper integration. Use it for the systems that are too old, too niche, or too locked down to expose APIs. For newer systems, continue pushing for API-native agents, connectors, and MCP servers. Over time, the balance will shift, and the desktop automation layer will handle a smaller but still important set of edge cases.

Bottom Line

GitHub Copilot computer use is the most practical step yet toward agentic work on the desktop. It does not require new integrations, it respects permissions and approvals, and it runs on the Windows and Mac machines people already use. For developers and knowledge workers drowning in GUI-only busywork, it is a genuinely useful new tool.

Pick one repetitive desktop workflow. Turn on computer use. Describe what you want. Approve the steps. Review the result. That single experiment will tell you more about the future of agentic work than any announcement ever could.