AI & Creative Tools

A Full Retrain Just Fixed the Hidden Number That Was Quietly Breaking an Open Image Model

MOSS-Image-heal, a from-scratch finetune by LyCORIS author KohakuBlueleaf, shrinks MOSS-Image-Base's largest internal activation value by nearly 19x — pulling it back from the edge of fp16's numeric ceiling without changing what the model draws.

Open image models usually get judged on what they draw. MOSS-Image-heal, published this week by KohakuBlueleaf — the developer behind the widely used LyCORIS fine-tuning library — is a rarer kind of release: a full retrain of an existing model that’s meant to change nothing about its output and everything about whether it can be trusted at half precision.

The number that was too big

Every transformer carries a “residual stream” — the running total of information that flows between layers, which each layer reads from and adds back into. In MOSS-Image-Base, a handful of channels in that stream were carrying enormous values: the single largest recorded activation hit 58,411. That’s a problem because fp16, the 16-bit floating-point format ComfyUI users run models in to save VRAM, tops out at 65,504. A value that close to the ceiling is one unlucky input away from overflowing into inf and corrupting a generation — the kind of failure that looks like a random crash or garbage output with no obvious cause.

KohakuBlueleaf’s fix wasn’t a patch or a clamp bolted on afterward. MOSS-Image-heal is a complete finetune of MOSS-Image-Base’s transformer, with the training process itself penalizing those oversized channels — “stream healing,” in the model card’s own terms. The result: the largest activation value drops to 3,083, a 7.5x reduction in the stream’s overall RMS, and every weight element now stays within 64x of its tensor’s own RMS (the base model had 349 elements that didn’t). Headroom to the fp16 maximum goes from barely more than the value itself to over 21x.

What didn’t change

The point of a “heal” rather than a new model is that the output stays recognizably the same model. KohakuBlueleaf’s own probe-set numbers back that up: flow loss moves from 0.4280 to 0.4289, and velocity cosine similarity between base and healed outputs sits at 0.9983 — close enough that the model card frames this explicitly as a numerics fix, not a capability change. Everything outside the transformer — text encoder, tokenizer, VAE, scheduler, and the surrounding pipeline code — is untouched, taken straight from the official MOSS-Image-Base release.

The release ships in the formats ComfyUI users actually load: bf16 and fp16 safetensors (~10.2GB each), plus an int8 ConvRot quantization at 5.4GB using the same layout as Comfy-Org’s existing int8_convrot files, so it drops into an existing MOSS-Image workflow without a loader change.

Part of a pattern, not a one-off

KohakuBlueleaf has spent the past few weeks building out a small ecosystem of MOSS-Image tooling: KohakuMoss, a PyPI package released October 18 that trains LoRA, LyCORIS, or full finetunes on MOSS-Image-Base from cards with as little as 6GB of VRAM, and KohakuMossTrainer, the GitHub repo documenting the underlying concepts — massive residual channels, what causes them, and how a stream-healing finetune shrinks them — that this release is built on. Numerics fixes like this one rarely make headlines the way a new model launch does, but for anyone running MOSS-Image at fp16 in production, a model that’s measurably further from silently overflowing is the more useful release of the two.