Vercel Labs publishes an Apache-2.0 CLI that lets agents drive a desktop

single source· 1 articles · confidence: medium · first seen 2026-10-31 10:10 UTC

What this means for you

If you are automating a desktop app that has no API, look here before writing pixel-based clicking yourself: the tool addresses named controls rather than screen coordinates, which the project presents as the more stable route. Nothing has been published on accuracy or reliability, so test it against your own applications before depending on it.

Vercel Labs released agent-desktop, an Apache-2.0 command-line tool written in Rust that lets an AI agent operate a desktop on macOS, Windows and Linux. Rather than taking screenshots and guessing where to click, it reads the operating system's accessibility tree — the structured list of on-screen buttons, fields and menus that screen readers already consume — and acts on stable references to those controls. That should make repeated actions cheaper to run, because a control keeps its reference when it moves on screen. It is a source repository, not a hosted product: no pricing, no published evaluation and no support commitment accompany it.

Key facts

  • ·Vercel Labs released agent-desktop, a command-line tool for giving AI agents control of a desktop, announced on 31 October 2026. source
  • ·The tool is licensed under Apache-2.0. source
  • ·It is written in Rust. source
  • ·It targets macOS, Windows and Linux. source
  • ·It reads OS accessibility trees and acts on stable references rather than pixel coordinates. source

What the sources say

  • MarkTechPost — Brief release note covering the licence, language and accessibility-tree approach to desktop control.

Sources

The original reporting. Follow these — they did the work.

← the wire