Computer Use in the Gemini API (Generally Available)

The Gemini API tool that lets an agent operate a browser, phone, or desktop by looking at screenshots is now generally available, for developers.

The model looks at a screenshot and returns the next action, such as a click, typing, scrolling, or opening a URL. Your own code carries out the action and sends back a new screenshot, and the loop repeats until the task is done.
Works across browser, mobile, and desktop environments, and you can add your own actions, such as handing control back to a person.
Risky steps like payments, sending messages, creating accounts, or accepting terms come back flagged for confirmation or blocked.
An opt-in check can stop the agent when a screenshot contains hidden instructions meant to hijack it.
Available in both the Interactions and GenerateContent APIs. Supported models include Gemini 4 Argon and Gemini 3.8 Flash, which Google recommends.

Automating repetitive work in web apps that have no API, like form filling or pulling data across several sites.

A task lives entirely inside a website or app screen and you want an agent to click through it for you.

Developers and ops-minded teams building agents for marketing or reporting tools without integrations.

Join the newsletter to get ahead with AI.

I share strategies, new modules, free guides, and tools to get you better at using AI in your work.