Feed a large language model a scoring function and let it iteratively rewrite a population of programs, and it will — as recent systems have shown in mathematics, algorithm design, and scientific computing — produce genuinely striking results. It will also, almost always, run as a fully automated black box: hundreds of candidate programs, scores climbing, and no real interface for a human expert to understand why, let alone redirect it.
EvoMaestro, from Feng Liang, Sizhe Cheng, Yikai Li, Ruijie He, Xiaolin Wen, and Yong Wang, names this gap “semantic oversight” — the need to understand and steer populations of evolving algorithmic ideas, not just monitor their fitness scores. The paper, accepted to ACM UIST 2026, grounds the problem in a formative study with 8 domain experts, which produced six design requirements for what an oversight interface actually needs to do.
The system itself is an interactive visual analytics tool built on a steerable evolution framework: it helps experts locate and compare noteworthy programs in a population, guide further evolution with natural-language instructions, combine ideas from separate promising branches, and prune directions that aren’t paying off — turning “watch the automated process run” into something closer to an actual collaboration.
The evaluation is where the claim gets tested properly: a seven-day system demonstration showing how an expert used these capabilities across a long-running evolution process, followed by a within-subjects study with 12 participants that found EvoMaestro improved understanding of the evolution process, reduced cognitive workload, and supported real steering rather than passive observation. The authors’ framing is worth keeping: as LLM-driven search gets more capable and more autonomous, preserving expert agency isn’t a nice-to-have interface layer bolted on afterward — it requires support for both informed judgment and the ability to act on it, designed in from the start.