UseDesktop Research

DesktopBench: Benchmarking Agents on Economically Valuable Professional Software Work

Seungju Chae

UseDesktop

Paper Code Data Leaderboard

Paper, code, data, and public ranking artifacts will be attached as DesktopBench moves from evaluation preview to public release.

Abstract

DesktopBench is a benchmark for agents working in professional software workflows across commerce, accounting, support, sales, and office tools. It is particularly designed to test whether an agent can complete economically valuable work end-to-end.

Each workflow includes a resettable RL environment, tasks, and a programmatic verifier. This makes the benchmark useful for evaluation, reproducible rollout testing, and future training runs where success can be checked from the final state rather than only from a transcript.

Demo videos

Agent runs on the same workflow contract

These cards are reserved for public videos once trace bundles and scoring artifacts are ready.

Run video pending

Opus 4.8

Commercial frontier model

A full desktop workflow run with screen trace, action log, final state diff, and verifier result.

Run video pending

OpenAI 5.4 mini

Small frontier model

A lower-cost agent attempt on the same resettable workflow contract and scoring surface.

Run video pending

RLVR open-weight model

Trained open-weight run

A policy improved against DesktopBench-style rewards, evaluated with the same verifier.

Benchmark

A benchmark for real desktop work

Browse suites
200

Core tasks

The initial release target across commerce, accounting, support, and office workflows.

4

Core suites

StoreOps, AccountingOps, SupportOps, and OfficeOps form the first public target.

8

Total suites

The broader map expands into sales, legal, HR, and public portal workflows.

Env + task + verifier

Task contract

Every workflow is packaged as a resettable RL environment, tasks, and a programmatic verifier.

Resettable environment

Every task starts from a known seed and can be replayed across model attempts.

Executable verifier

Tasks are scored by final state, artifact, and side-effect checks rather than screenshots alone.

Real desktop work

Suites are modeled after store, accounting, support, sales, office, legal, HR, and public portal operations.

Optional trajectory

Human demonstrations and failed attempts can be attached as training or audit evidence.

Results

Benchmark results

Open leaderboard
Model Track Verified success Cost Evidence
Opus 4.8 DesktopBench-Core TBD TBD Awaiting public trace bundle
OpenAI 5.4 mini DesktopBench-Core TBD TBD Awaiting public trace bundle
RLVR open-weight model DesktopBench-Core TBD TBD Awaiting public trace bundle

Task examples

Economically valuable workflows become executable tasks

StoreOps-50

StoreOps

Open

Fulfill order #1048 using tracking number CJ123456789KR and carrier CJ Logistics.

order.status == fulfilled, carrier == CJ Logistics, tracking_number == CJ123456789KR

AccountingOps-50

AccountingOps

Open

Match the KRW 1,200,000 bank transaction from Acme Korea to invoice INV-1028.

txn_8842.matched_invoice == INV-1028 and txn_8842.reconciled == true

SupportOps-50

SupportOps

Open

Classify ticket #5831 as Billing and set priority to High.

ticket.category == Billing and ticket.priority == High

OfficeOps-50

OfficeOps

Open

Find the latest email from [email protected] and save the attached invoice PDF to Invoices/AWS.

file exists in Invoices/AWS and source email/message id matches latest AWS billing email

Evaluation

Task contract

task_id Stable identifier such as StoreOps.OrderFulfill.001.
suite / family Benchmark suite and task family for stratified splits.
difficulty L1-L5 long-horizon difficulty level.
instruction Natural language task objective given to the agent.
environment Mock app or desktop workspace runtime.
initial_state_seed Seed used to reset state before each rollout.
allowed_apps Browser, files, spreadsheet, email, calendar, or multi-app desktop surface.
success_verifier Executable final-state, artifact, or process checks.
side_effect_verifier Checks that unrelated records/files were not changed.
limits Max steps and max minutes for the rollout.
trajectory Optional human demonstration or known-good trace.
reward Success and partial-credit reward contract for RLVR.

Citation

Cite DesktopBench

@misc{chae2026desktopbench,
  title={DesktopBench: Benchmarking Agents on Economically Valuable Professional Software Work},
  author={Chae, Seungju},
  year={2026},
  note={Technical report in progress}
}