The reason most creative tools that use a language model send your work to somebody else’s server is memory. A capable model in full precision wants dozens of gigabytes of VRAM, which rules out laptops, installations, and anything that has to run in a gallery without a network connection.
PrismML’s Ternary-Bonsai-2-27B is the most aggressive answer we’ve seen to that, and it arrived hard: created September 16, it entered our tracker at trending rank #4 with 405,609 downloads and 615 likes.
What ternary means
Standard weights are 16-bit numbers. Ordinary 4-bit quantization rounds them to one of sixteen values. Ternary rounds them to one of three: −1, 0 or +1, with a shared FP16 scaling factor per group to restore magnitude.
That sounds like it should destroy the model, and the interesting claim is that it doesn’t. Packed densely, it works out to about 1.72 bits per weight. Two formats ship: PTQ1_0 (dense trits, 1.75 bpw) and PQ2_0 (trits in 2-bit slots, 2.13 bpw).
The result:
- 5.95GB (PTQ1_0) or 7.21GB (PQ2_0) for the language model
- against a ~54GB FP16 baseline — roughly a 9× reduction
- with an optional vision tower (~0.63GB) that loads only when you feed it an image
The performance claim, and how to read it
PrismML reports 84.78 average across 14 thinking-mode benchmarks, which they characterise as retaining 98.2% of FP16 intelligence, with maths at 96.57 against a 97.06 baseline and coding matching at 89.42.
Treat that the way we’d treat any self-reported figure: it’s the vendor’s own evaluation of their own model, on benchmarks they selected. The direction is credible — ternary and other extreme-quantization work has been converging on “much smaller, slightly worse” for a while — but 98.2% retention at 1.72 bits is a strong claim that deserves independent replication before anyone treats it as settled.
The model is derived from Qwen3.8-27B and uses a hybrid attention scheme (75% linear, 25% full), so the compression isn’t doing all the work on its own.
Speed, and the catch
Reported throughput: ~28 tokens/s on an Apple M5 Pro, ~130 tok/s on an RTX 5090, ~30 tok/s on a 72W L4. The M5 figure is the one that matters for this site — that’s a laptop, unplugged, running a 27B model with vision at readable speed.
The catch is significant: it needs PrismML’s own llama.cpp fork. Stock llama.cpp cannot read these formats. So anything you build on it depends on a fork tracking upstream, and if that stops, your tool stops. Licensing is Apache 2.0, which at least means the work can be carried on by others.
Why it’s here rather than in a machine-learning newsletter
We cover local models because offline, private and free-to-run is what installations, instruments and studio tools actually need. A 6GB model with vision that runs on a MacBook changes the calculus for a gallery piece that has to describe what a camera sees, an instrument that responds to spoken instruction, or a studio tool that never uploads a client’s material.
The honest position today: promising numbers, a vendor-run benchmark, and a fork dependency. Worth testing this week, worth waiting on before shipping.