Most AI image-editing benchmarks score results with another model acting as judge. EditAgent, a new paper from Adobe Research and UC San Diego posted to arXiv on October 29, instead put the work in front of five working image-editing professionals and asked them to blind-rate the output — and the gap between “technically completed” and “a professional would accept this” turns out to be large.
The headline number
Across 600 image-editing tasks, a general-purpose editing model (Flux-2-Klein-9B) produced no usable result in 42.7% of cases. Routing those same failures to specialized tools — dedicated sharpening, rotation, or zoom operations rather than one model trying to do everything — recovered roughly two-thirds of them. The paper’s practical conclusion is straightforward: a single generalist model asked to handle every kind of edit is still the weak point, and task-specific tools remain meaningfully better at their narrow jobs than a generalist agent trying to cover all of them.
A second finding cuts against how most editing agents are built. Professionals preferred parallel variants — several independent attempts generated from the original image at once — over an agent iterating on a single edit, in 78.5% of head-to-head comparisons where they expressed a clear preference. Parallel generation also required about half as many evaluation passes to reach an accepted result, suggesting that iterative refinement (the more common agent design) is not just slower in practice but produces a worse final result on average.
Building a better judge
Because human rating doesn’t scale, the paper also tests which automated judges best approximate professional taste. Qwen3.8-27B picked the same “usable” result as professionals 81.3% of the time, beating a model fine-tuned specifically for this job (EditScore-72B, 74.8%). Giving judge models extra verification context — close-up crops, change maps, and measured image statistics alongside the raw before/after — raised accuracy by 6.3 to 9.3 points across all seven language models tested, a reminder that better automated evaluation often comes from richer input rather than a bigger model.
The resulting EditAgent system, combining these findings, outperformed seven other AI editing agents across three benchmarks including EdiVal — though at roughly ten times the compute cost of a single-model editor. The team’s ~26,000 verified training examples, generated through this evaluation process, also improved fine-tuned editors’ scores across all three benchmarks when used as training data, independent of the agent itself.
Adobe added AI agents to Photoshop, Premiere, and other Creative Cloud apps in a public beta starting in June; this paper is research rather than a shipped feature, though it points at where that kind of agent still breaks in practice.