Research & Innovation

AVAtt Tracks Where Eyes Actually Go on a Painting — Free-Viewing, No Task

An eye-tracking dataset and benchmark for visual attention on paintings, extending work published at ETRA 2024. The interesting choice is free viewing: no question to answer, no object to find, just looking.

AVAtt: A Free-Viewing Eye-Tracking Dataset and Benchmark for Visual Attention on Paintings was submitted to arXiv on 21 October 2026 (arXiv:2610.30481), filed under computer vision, human-computer interaction and image processing.

Its Comments field is important and we are going to lead with it:

Extenstion of https://doi.org/10.1145/3649902.3655656 published at ETRA 2024

So this is not a new result announced from nothing. It is an extension of a paper the same line of work published at ETRA 2024 — the ACM Symposium on Eye Tracking Research and Applications. The arXiv posting is the extended dataset and benchmark; the core was peer-reviewed two years ago. That is a legitimate and common way for a dataset paper to grow, and it is exactly the kind of thing our Comments-field habit exists to catch — not because it disqualifies the work, but because “new dataset” and “extension of a 2024 dataset” are different claims and only one of them is true here.

Why paintings are a special case

Computational saliency — predicting where a person will look in an image — is a mature field with a problem: its training data is photographs, and much of its ground truth comes from task-driven viewing. Participants are asked to find something, judge something, or answer a question, and their gaze is recorded while they do it.

Paintings violate both conditions in useful ways.

They are composed to direct attention. A painter has spent effort arranging where your eye goes first, where it rests, and how it travels. Leading lines, value contrast, the single warm note in a cool field, a face turned toward or away — these are attention-engineering techniques developed over centuries without any of the vocabulary that saliency research uses. A dataset of gaze on paintings is, among other things, a measurement of whether those techniques work on contemporary viewers.

And people look at them for no reason. Gallery viewing is the paradigm case of free viewing: nobody sets you a task, there is no correct answer, and how long you spend is entirely up to you. That is a different cognitive regime from “find the pedestrian in this dashcam frame,” and models tuned on the latter have no particular reason to predict the former.

AVAtt’s name encodes the choice — free-viewing attention — and the choice is the contribution as much as the data is.

What it is for

Two audiences, and they want different things from it.

For vision researchers, it is a benchmark: a held-out set on which to test whether saliency models trained on photographic, task-driven data generalise to composed, free-viewed images. The honest expectation is that they will do noticeably worse, and that the failure modes will be informative — if models over-predict faces and under-predict compositional voids, that tells you something about what the models have actually learned.

For artists and curators, it is empirical data on a question usually settled by assertion. Hanging decisions, lighting, label placement and the sequencing of a room are all attention arguments made without measurement. A public dataset of where people actually looked, with no task imposed, is a resource that side of the field has rarely had.

What we are not claiming

We have the title, the arXiv ID, the subject classifications, the submission date and the provenance note. We have not read the full paper, so we are not reporting the number of participants, the number of paintings, the eye-tracker used, the benchmark’s baseline scores, or which models it evaluates. Those are the figures that would let you judge whether the dataset is big enough to be a benchmark rather than a pilot, and they matter.

What we can tell you is that it exists and that it is an extension of peer-reviewed ETRA 2024 work rather than an unreviewed first appearance. We checked the provenance pointer as far as it can be checked: the DOI in its Comments field — 10.1145/3649902.3655656 — is registered and redirects to the ACM Digital Library, which blocks automated reads, so we have confirmed the reference resolves without being able to read the 2024 paper behind it. That is better than an unsupported claim of novelty and short of a verified lineage, and it is worth saying which one you are getting.

The one thing to take away

If you work with images and you use a saliency model, or a crop-suggestion tool, or an automatic thumbnail generator, or any of the attention-prediction machinery now embedded in creative software: it was almost certainly trained to predict task-driven gaze on photographs. Pointed at a composed image, viewed freely, it is operating outside its training distribution and has no way of telling you so.

A benchmark that measures exactly that gap is overdue.

Sources