Computer-Use Agents

Agents that see a screen (screenshots) and act with mouse and keyboard, instead of calling APIs. Two flavours: full-desktop control and browser-only control.

Vendor lineage

• Anthropic: computer use went into public beta on 2024-10-22 with Claude 3.5 Sonnet (API, Bedrock, Vertex AI). Docs today: the computer_toolset_20260801 tool, GA on the Claude API and Google Cloud, with 17 actions (screenshot, zoom, clicks, drag, scroll, type, key…). Supported: Fable 5.1/5, Mythos 5.1/5, Opus 5.5/5/4.8, Sonnet 5.5/5. Platform availability: GA on the Claude API and Google Cloud; beta on Claude Platform on AWS, Amazon Bedrock and Microsoft Foundry. The earlier computer_20251124 tool is deprecated; migrate to computer_toolset_20260801 on Claude 5.5+ models. Screenshots should be scaled to at most 2576 px on the long edge. Anthropic’s advice: use a sandboxed VM, allowlist domains, require human confirmation; classifiers flag prompt injection in screenshots but do not replace these.
• OpenAI: Operator (early 2025) → ChatGPT agent (2025-07-17) → removed summer 2026, replaced by ChatGPT Work with a separate cloud browser. The OpenAI help centre says ChatGPT agent is no longer available and points to Work, which has a separate cloud browser that can pause for sign-in or takeover (OpenAI Support, via forum thread; exact removal date July-August 2026 is disputed, see openai-operator). Atlas browser ended 2026-08-09. See openai-operator, chatgpt-atlas.
• Google (Google): Gemini API Computer Use is still labelled Preview (Gemini 3.x recommended incl. 3.8 Flash; 2.5 legacy). Browser, mobile and desktop environments; coordinates normalised 0-999; safety categories (payments, communications, account creation, legal) with require_confirmation. Google’s earlier browser agent Project Mariner is not covered here (its current status could not be confirmed).
• Open source: browser-use (MIT, ~117k stars on 2026-10-02; hosted Cloud with usage-based pricing and a signup credit; also a CLI and its own BU2 model per the README); browseros (AGPL-3.0, local MCP browser for agents); lightpanda-browser (headless, no rendering).

Consumer AI browsers with agents

perplexity-comet, dia, Edge “Browse with Copilot” (microsoft-edge-copilot), Opera Neon (opera), fellou, strawberry-browser.

Related

claude-agent-sdk, agentic-flow, ai-agent-platforms-comparison, openclaw.

Risks

Prompt injection from page content, credential exposure in logged-in sessions, irreversible actions. Keep a human in the loop for payments and messages.

Sources (accessed 2026-10-02)

• https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool
• https://www.anthropic.com/news/3-5-models-and-computer-use
• https://ai.google.dev/gemini-api/docs/computer-use
• https://github.com/browser-use/browser-use
• https://github.com/browseros-ai/BrowserOS
• https://community.openai.com/t/agent-mode-was-removed-with-no-real-replacement/1389601
• https://github.com/browser-use/browser-use (re-checked 2026-10-02)