Is Computer Use the Next Step for Coding Agents?
Coding agents are approaching a ceiling on coding tasks. What comes next may be computer-use agents that can complete work across applications and websites.
Technical articles, research notes, and thoughts.
Coding agents are approaching a ceiling on coding tasks. What comes next may be computer-use agents that can complete work across applications and websites.
Human behavior is not automatically a good policy. Five challenges in expert trajectory collection, from aligning continuous actions with discrete observations to failure recovery and environment decay.
How can we keep generating tasks beyond a model's current capabilities while ensuring they remain executable, verifiable, and automatically scorable? Lessons from WebFactory and environment-informed task synthesis.
Building a human data collection pipeline from scratch: 31,725 human trajectories and 318,000 interaction steps across 428 real websites.
GUI capability remains necessary for computer-use agents, but a complete agent should dynamically choose among APIs, CLIs, structured state, and graphical interaction.
Many systems appear to operate websites without actually using the browser through its GUI. WebChain addresses the data problem behind this capability gap.
WebFactory explores a central question: if language models already understand web content, why do they still struggle to complete tasks reliably in real GUI environments?
We will open source the largest real human trajectory dataset for web agents.
Agents are replacing traditional interaction paradigms, transitioning from passive response tools to active decision-execution systems. This raises new questions and directions for GUI agent technology itself and how we understand future human-computer interaction.
We need to strip away the semantic bubble of capital markets and return to the original proposition defined by reinforcement learning: how to achieve true Markov Decision Processes in uncertain environments.
The current AI wave is defined by 'content generation,' but the real paradigm shift is happening beyond 'content.' GUI Agent is not a new form of content output, but a Side Effect manufacturing system. It marks AI's transition from probabilistically 'describing the world' to deterministically 'changing the world.'