Grafy News
Try Grafy free

Google adds mac‑wide computer‑use agent to Gemini

Google’s Gemini app for macOS now can act on local files and desktop apps, letting users run multi‑step tasks.

Illustration for the news story: Google adds mac‑wide computer‑use agent to GeminiSource: The Verge — AI

What is the new Gemini computer‑use feature?

Google announced that the Gemini app for macOS now includes a computer‑use capability. The company describes it as an ability for Gemini to "get multi‑step tasks done across your local files, desktop apps, and the web." In practice, the assistant can open files, type into forms, manipulate windows, and pull information from the internet without the user manually performing each step. The rollout begins in English for users in the United States.

How does it work on macOS?

When the feature is enabled, Gemini displays a separate background window that streams a live preview of the actions it is taking. Users are prompted to grant the assistant access to specific applications, giving them granular control over which programs the AI can control. The preview window appears inside the chat pane, so the conversation and the on‑screen activity stay linked. According to The Verge, the permission UI was introduced together with the macOS 27.3 beta, aligning with Apple’s upcoming restrictions on apps that request full‑disk access.

What tasks can it automate?

Google cites several practical scenarios. Gemini can fill out web forms by pulling data from local spreadsheets, assemble slide decks by pulling images and text from a folder, and generate product listings by extracting details from inventory files and publishing them to e‑commerce sites. Because the assistant can switch between local applications and the browser, it can stitch together workflows that previously required manual copy‑and‑paste or repetitive clicking. Creators can, for example, ask Gemini to draft a newsletter, pull the latest metrics from a spreadsheet, and export the result as a PDF—all in a single conversational exchange.

Why is Google releasing it now?

The timing coincides with Apple’s plan to tighten "full‑disk access" rules for macOS apps, citing security concerns around AI agents that can manipulate the file system. By shipping its own permission UI ahead of Apple’s policy change, Google positions Gemini as a compliant alternative that gives users visibility into what the model can do. The Verge notes that the feature is being rolled out before the new restrictions take effect, suggesting a strategic move to capture early adopters who need AI‑powered automation while still meeting Apple’s forthcoming guidelines.

How can creators and developers experiment with similar agents in Grafy?

Grafy does not yet offer a mac‑wide computer‑use agent, but the Grafy Chat app lets users converse with a range of AI models side by side, including those that support tool‑calling or function execution. By chaining prompts, creators can simulate multi‑step workflows: for instance, ask a model to generate text, then hand that output to a code‑generation model that writes a small script, and finally run the script in Grafy Code. The Graph app can map each step of such a workflow, visualising how data moves from one model to the next. While the exact desktop‑control capability of Gemini is not replicated, Grafy’s modular environment gives developers a sandbox to prototype similar automation pipelines and test permission‑style gating before macOS restrictions become mandatory.

Frequently asked questions

Does Gemini need full‑disk access to run on a Mac?

No. Google’s implementation asks users to select individual apps that Gemini may interact with, rather than granting blanket full‑disk access. This granular permission model aligns with Apple’s upcoming security guidelines.

Can Gemini operate on files stored in the cloud?

Yes. The announcement mentions that Gemini can work across local files, desktop apps, and the web, which includes cloud‑based services accessed through a browser. The assistant can retrieve data from online spreadsheets or cloud storage as part of a multi‑step task.

How do I know what Gemini is doing during a session?

Gemini opens a background window that shows a live preview of its actions. The preview appears alongside the chat, letting users watch the assistant navigate apps, fill forms, or move files in real time.


Source: The Verge — AI — Gemini on Mac can now use your computer