← Back

Computer-Use GUI Agent: Post-Training and Self-Evolving Evaluation

March 1, 2026
GUI agent post-training pipeline and self-evolving evaluation loop
Top: four-stage post-training, SFT → offline step-RFT → online GiGPO → multi-platform MOPD. Bottom: the trace evaluation → benchmark self-evolution loop.

Problem

Automating business workflows on web and desktop interfaces means a GUI agent has to search, fill forms, move across pages, and submit results over many steps, with high demands on action correctness, error recovery, and long-horizon stability — a base model is far from usable out of the box. At the same time, task types, page states, and action paths keep changing, so manual task-by-task review is expensive and slow to reveal capability gaps. The project therefore has two halves: training the agent (post-training) and measuring it reliably (automatic evaluation with a self-evolving benchmark).

1. Post-training: SFT → offline step-RFT → online GiGPO → multi-platform MOPD

Multi-source SFT trajectories and mixing. Four trajectory pipelines — teacher trajectories, switch-policy correction trajectories, hint-guided rollouts (rewritten into CoT afterwards), and rejection sampling — with mixing experiments to choose the data blend. Teacher-only SFT alone gives a 9.7% gain.

Switch-policy correction trajectories. A weak policy first takes a wrong action, then a strong policy takes over and corrects it, covering fail-and-recover situations. The loss is masked on the erroneous steps that were later corrected, so the model does not imitate them:

where is the interaction history and the weak-policy steps that were subsequently corrected.

Offline step-level GRPO (step-RFT). With the trajectory context frozen, candidates are sampled for a single atomic action and optimized with GRPO, using the group-normalized advantage from DeepSeekMath:

A rule-based verifier scores format, action type, and arguments in three stages; a click earns 1 inside the golden box and decays with distance outside it (schematic):

Initialized from the SFT checkpoint with the highest share of correction trajectories, step-RFT adds a further 4%.

Verifiable fine-grained reward. Task progress is the main term, with penalties for side effects, false completion, aborting after success, and overrunning the step budget (schematic):

A false-completion correction on wrong AnswerSheet answers stops the agent from collecting reward by merely claiming to be done.

Online GiGPO two-level advantages + DAPO clip-higher. GiGPO normalizes at two levels. At the episode level, the trajectories of the same task form a group:

At the step level, steps from different trajectories that share the same state — identified here by the action prefix — form a group, scored with the discounted return :

The policy update uses DAPO’s asymmetric clipping (clip-higher, ), leaving room for low-probability but useful actions to grow:

Compared with trajectory-level GRPO, the two-level advantage gives much finer-grained credit assignment.

Multi-platform MOPD: merging platform specialists into one agent. Operating systems differ widely in UI elements, shortcuts, and interaction conventions, and mixing every platform into one RL run makes them interfere. So after GiGPO, a separate GiGPO specialist is trained for each of macOS, Linux, Windows, and Web — — and MOPD (Multi-Teacher On-Policy Distillation) distills them into a single student. The student samples from its own distribution, and every token is supervised with reverse KL by the specialist that matches the task’s platform, :

In practice the student optimizes a surrogate with training–inference importance sampling, where is the rollout policy on the inference engine:

The token-level advantage is the specialist’s log-probability ratio plus an outcome-reward advantage — here the verifiable reward and GiGPO advantage from the previous stage:

The student thus absorbs each specialist’s platform-specific behavior while keeping task completion as the global objective, ending with one model that covers all four platforms.

2. Automatic evaluation and a self-evolving benchmark

A closed loop of trace evaluation → benchmark self-evolution: each task is decomposed into atomic scoring points, judged one by one by an LLM-as-Judge, and routed to a simple or a complex evaluation path by task type.

  • Windowed point judging and complex-task checks. The trace is cut chronologically into evidence windows; points are judged in batches per window and aggregated by group. For information-seeking tasks, only page facts the agent actually observed are extracted, catching answers that are right but whose evidence was invented. For looping tasks, the entities acted on are extracted per window, deduplicated across steps, and the execution count is checked, avoiding both missed and double counts.
  • Benchmark self-evolution. Four mutation operators turn seed tasks into harder ones along the solution chain; capability constraints built from real-device traces keep every new task achievable. Candidates are written into the benchmark only after real-device reachability checks and independent review, so the benchmark keeps growing on its own.

Results

MetricResult
Success on the 200-task expert-solution benchmark31.3% → 49.2% (SFT → step-RFT → GiGPO)
Gain from teacher-only SFT+9.7%
Gain from step-RFT+4%
LLM-as-Judge agreement with human labels94.3% (complex tasks 83.2%)

Formula references: GRPO from DeepSeekMath (Shao et al., 2024), GiGPO from Feng et al. (2025), MOPD from the MiMo-V2-Flash technical report (Xiaomi, 2026), and DAPO from Yu et al. (2025). Equations marked “schematic” are structured descriptions of this project’s design, with coefficients omitted.