If you have ever designed a DSL so a model would stop writing buggy JavaScript, this paper is about your assumption.
Shorter Is Not Cheaper was posted to arXiv on 23 October 2026 (arXiv:2610.32129) by Jiantang Huang, a single author. The Comments field gives 15 pages, 6 figures and 8 tables, and names no venue, so this is an unreviewed preprint.
The expectation being tested
Chat assistants increasingly answer conceptual questions with animated explanations. There are two ways to produce one:
- Free-form code — the model writes HTML and JavaScript, which runs.
- A declarative specification — the model emits a structured description that a fixed runtime plays.
The declarative route is widely believed to be better on every axis. The paper states the belief exactly as practitioners hold it:
The declarative route is widely expected to be cheaper, faster and more reliable, because its outputs are shorter and constrained.
The reasoning feels airtight. A spec is shorter than a program. Fewer tokens is less money and less latency. A constrained grammar has fewer ways to be wrong. Each step is plausible and the conclusion is wrong often enough to be worth measuring.
How it was done, which is the best part
Three design decisions here are better than most work in this area.
The hypotheses and analysis plans were frozen before the runs. Pre-registration, in effect. In a field where it is normal to run an experiment, notice an interesting pattern and build the framing around it, committing to the analysis in advance is what makes a negative result mean something.
They built both sides properly. The declarative format is ExplainSpec, described as time-indexed, designed for this comparison rather than borrowed. A weak DSL would have rigged the result.
An observation contract, so grading is mechanical. Every generated explanation is “programmatically inspectable”, scored by validated, pacing-agnostic checkers. Pacing-agnostic matters: an animation that conveys the same content more slowly should not be marked down for timing, and a human grader would struggle to hold that line consistently.
The scale: 20 explanation requests from five domains. Three Claude models, 360 jobs. Then a follow-up with the open-weight Qwen3-14B, with thinking enabled and disabled, 240 jobs. Every request answered in both formats. 600 jobs in total.
The results
The specifications were shorter, every time. 25–48% shorter in answer tokens — where “answer tokens” means output excluding reasoning — for every model.
And then:
- With reasoning enabled, total output fell significantly for only the largest Claude model. For the others, the saving in answer tokens was absorbed somewhere else.
- No model generated significantly faster. Not one.
So the shorter artefact did not produce a cheaper or quicker job. The abstract notes that the reasons differed between the Claude models and Qwen, and the arXiv record truncates before explaining how, so we will not speculate.
Why “answer tokens” is the word to notice
The mechanism this measures is one that anyone budgeting an LLM feature should internalise.
Output length is not the bill. Total generated tokens are the bill, and on a reasoning model a large and variable share of those are spent before the answer starts. If a constrained format makes the model think harder — because it must now plan within a grammar, map intent onto a fixed vocabulary, and get the structure right first time rather than writing its way toward a solution — then reasoning expands to fill the space that the shorter answer freed.
That trade is invisible if you measure the artefact instead of the job, which is exactly what “the output is shorter, so it must be cheaper” does.
The latency finding follows the same logic and is the more practically annoying one. Users do not experience answer tokens; they experience time to a working animation. A format that halves the answer and leaves the clock alone has, from the user’s seat, achieved nothing.
What it does not tell you
Reliability is not in the abstract’s headline findings. The expectation being tested had three parts — cheaper, faster, more reliable — and the quoted results address the first two. A constrained grammar may well still produce fewer broken animations, which for a shipped product can justify the format on its own regardless of cost. Read the paper for the checker scores before concluding either way.
One author, no venue, no review. Treat it as a well-designed experiment that has not yet been scrutinised.
ExplainSpec is the paper’s own format. A fair design, since a deliberately weak DSL would have been worse, but it is one point in a large space. Someone else’s format might behave differently.
The takeaway for people who build these things
Three things worth carrying into your next design review.
Measure the job, not the artefact. Count total tokens including reasoning, and wall-clock time to a rendered result. If your justification for a DSL is output length, you have not measured the thing you are paying for.
Constraint moves work, it does not delete it. Composing within a grammar is cognitive labour you have relocated from the output into the planning. Sometimes that is a good trade — it is where reliability gains come from — but it is a trade, not a saving.
Keep the free-form baseline running. Pre-registering a comparison, as this paper did, is the cheap version of the discipline: decide in advance what would make you abandon the DSL, then look.
Sources
- Shorter Is Not Cheaper: A Controlled Comparison of Declarative Animation Specifications and Free-Form Code for LLM-Generated Explanations — arXiv:2610.32129
- arXiv PDF
- arXiv listing for cs.HC
- Qwen3-14B — Hugging Face