Step-level Optimization for Efficient Computer-use Agents

This research highlights the inefficiency of current computer-use agents, which overuse large multimodal models for every GUI interaction. It argues that tasks are heterogeneous, with routine steps needing less compute, while errors concentrate at high-risk moments like stalls or semantic drift, requiring targeted optimization.