Nearly every text-to-image model you have used works in a latent space. A variational autoencoder compresses the image roughly 8× per axis, the diffusion transformer learns in that compressed space, and a decoder expands the result back to pixels. It is enormously cheaper, and it is lossy.
Iris-3B, from SperidLabs, does not do that. The model card’s claim is blunt: a 3-billion-parameter model that “paints every pixel directly — no VAE, no latent space.” The stated motivation is that nothing is lost to a lossy, texture-biased latent.
The interesting part is what the project’s own paper concluded about whether that helps.
The architecture
- 3B parameters, plus a frozen 4B text encoder (Qwen3-VL-4B-Instruct)
- A diffusion transformer with 8 dual-stream and 16 single-stream blocks
- A 4-block pixel head that converts each 16×16 patch back into pixels
- Weights stored as F32 safetensors; the base model is about 12 GB
- Apache 2.0, text encoder included under its own Apache 2.0 terms
Training used rectified flow from scratch on a resolution curriculum — 256, then 512, then 1024 px — followed by supervised fine-tuning at 1024 px, for 665K steps in total. Default output is 1024×1024, roughly one megapixel; anything larger needs the upscaler.
Because no architectural changes were needed, the same network fine-tunes into a general vision model: monocular depth estimation, image restoration and upscaling ship as separate heads in the repo.
The negative result, stated by the authors
The paper is Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning (arXiv 2610.09450, submitted 7 October 2026), by Hanqiu Li Cai and Chema Garabito of SperidLabs.
They tested the hypothesis properly. Alongside pretraining Iris-3B in pixel space, they also converted the latent FLUX.2 Klein base 4B model to pixel space, then fine-tuned both families for depth estimation and for restoration/super-resolution. The findings:
- No significant benefit from a pixel-space generative prior
- On depth estimation, Iris-3B performs on par with latent FLUX.2 Klein; the converted pixel model falls behind
- On 4× DIV2K restoration, neither pixel model beats a latent FLUX.2 Klein fine-tune
What the work does establish is that pixel-space pretraining scales to 3B parameters and reaches text-to-image quality competitive with latent models — matching Qwen-Image on OneIG at 1024².
So the headline on the model card and the headline in the abstract point in different directions, and the abstract is the more careful of the two. A lab publishing the result that undercuts its own framing is worth more than the framing.
Turbo, and the trade it makes
The repo includes Iris-3B Turbo, a 4-step distilled version at about 6 GB. It is a full fine-tune of all 3B parameters, not a LoRA, distilled with Trajectory Distribution Matching, with CFG baked into the weights. The card reports about 0.7 s for a 1024×1024 image on one H100 at batch size 4.
The benchmark picture is genuinely mixed rather than a straight win:
| Benchmark | Base | Turbo |
|---|---|---|
| GenEval | 0.798 | 0.820 |
| DPG-Bench | 86.52 | 86.68 |
| LongText-Bench | 0.857 | 0.804 |
| OneIG-Bench | 0.540 | 0.516 |
Turbo is slightly ahead on prompt-following benchmarks and clearly behind on dense text and on OneIG. The card is explicit about Turbo’s other costs: less seed variety, possible grain on flat backgrounds, and a fixed 4-step schedule with no CFG and no negative prompt.
The other two heads
- Monocular depth: reported AbsRel 0.071, δ1 0.946, zero-shot, under the Marigold V2 protocol. Output is affine-invariant, not metric — relative depth, not distances in metres.
- Restoration and 4× upscaling: reported LPIPS 0.292, PSNR 20.68 on DIV2K validation. Large outputs are tiled into overlapping 1024×1024 windows.
Running it
git clone https://github.com/speridlabs/iris-3b.git
cd iris-3b
pip install -e . # Python 3.11+, PyTorch 2.7.1+
hf download speridlabs/iris-3b --local-dir iris-3b \
--exclude "depth/*" "upscaler/*" "turbo/*"
python scripts/sample.py --checkpoint iris-3b \
--prompt "a red fox sleeping in fresh snow, golden hour"
Default sampling is CFG 3 at 100 steps. --seed, --negative-prompt and --txt-file for batches are available. An NVIDIA GPU with CUDA is required, and the text encoder downloads on first run.
The --exclude flags are worth keeping: they skip the depth, upscaler and turbo folders, which you do not need for plain generation.
Caveats the card states itself
Text rendering and exact object counts are not always reliable, and outputs can be inaccurate, biased or unsafe and need review. A community 4-step turbo LoRA built on DMD2 distillation, credited to pentacoxian-dev, exists alongside the official turbo.
The repo currently sits at 313 likes against 1,494 downloads — the shape of a release people are watching rather than running, which is what a 12 GB F32 checkpoint with a CUDA requirement tends to produce.
Sources
- speridlabs/iris-3b — Hugging Face
- Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning — arXiv 2610.09450
- speridlabs/iris-3b — GitHub
- SperidLabs