Our tracker picked this up on the ratio rather than the volume: 302 likes against 630 downloads, six days after release. openbmb/MiniCPM-V-4.7-35B-A3B was created 7 October 2026, and the claim on the card is specific enough to be checkable:
13+ tokens per second, in 3 GB of memory, on an iPhone 17 Pro Max — from a model with 35 billion total parameters.
Why MoE on a phone was supposed to be impossible
A mixture-of-experts model activates a small fraction of its weights per token — here 3B active out of 35B, hence 35B-A3B. That makes the compute small. It does not make the memory small, because you do not know in advance which experts a token will need, so conventionally all 35B of weights have to be resident.
On a server with 80 GB of VRAM that is fine. On a phone it is the entire problem. MoE has therefore been a datacentre architecture: cheap to run, expensive to hold.
Three techniques together are what the card credits for getting around it:
Lookahead Expert Scheduling (LES). Published alongside the model. The name describes the mechanism — predict which experts upcoming tokens will route to, and stage them ahead of need, so the working set is a fraction of the full model rather than all of it. This is the load-bearing piece, and it is the first of the three that is genuinely new.
Fine-grained layer-wise quantisation. Different layers at different precisions rather than one uniform setting, spending bits where the model is sensitive.
DSpark speculative sampling integration. Standard speculative decoding — a cheap draft model proposes, the real model verifies several tokens at once — which raises throughput without changing output.
The claim is that these combine to give “lossless, smooth on-device inference” — the first open-source MoE model to do so. That is a strong claim and the figure attached to it is the one to test: 3 GB is a working set, not 35B of weights, so what LES is really being credited with is keeping the resident experts small enough to fit.
The efficiency framing is the interesting argument
The card makes a point worth separating from the benchmark numbers:
As model capabilities continue to advance, inference efficiency has become an important measure of a model’s value. On resource-constrained devices, delivering the strongest possible capabilities within a limited resource budget calls for higher capability density.
And the comparison they draw: in Instruct mode, 35B-A3B achieves a higher overall score in less inference time than Qwen3.6 35B-A3B — a same-shape competitor, which is the right comparison to make.
“Capability density” as the metric rather than capability is the correct frame for anything running on a device somebody is holding, and it is the frame almost no model card uses.
The vision side
Built on the intra-ViT early compression technique from LLaVA-UHD v4: visual tokens are compressed early inside the vision encoder, so subsequent encoder layers do less work.
The detail worth knowing for creative use is mixed 4×/16× visual token compression, which lets you trade visual detail against inference cost per task. A model that can be told “this one needs fine detail, that one does not” is considerably more useful than one with a fixed compression ratio — captioning an archive and reading text out of a screenshot have very different requirements.
It handles single-image, multi-image and video understanding.
Platform support is unusually broad
iOS, Android and HarmonyOS, with open-source on-device deployment support, plus day-zero support for chip platforms from Rockchip, Houmo, Intel, AMD and Arm.
Rockchip matters more than it sounds. Rockchip SoCs are what is inside a large share of cheap single-board computers and embedded displays — the hardware installation work actually runs on. Day-zero support for it means a local vision-language model in a gallery piece, on a board that costs tens rather than thousands, without a GPU.
Inference through SGLang, vLLM, llama.cpp and Ollama; fine-tuning through SWIFT and LLaMA-Factory; quantised variants in GGUF, BNB, AWQ and GPTQ, with FP8 added. Apache 2.0.
They also release Self-Replay Fine-Tuning, a method for extending capabilities while reducing forgetting — which is the problem that makes fine-tuning a multimodal model risky in the first place.
What to be sceptical about
“Lossless” is a strong word and the card does not define it. Lossless relative to what — the unquantised 35B-A3B at full precision, or the same model on a server? Layer-wise quantisation and speculative sampling are both nominally output-preserving in theory and not always in practice.
One phone, one number. 13+ tokens/s in 3 GB on an iPhone 17 Pro Max is a single measurement on current flagship hardware. What it does on a two-year-old Android, or on the Rockchip board it claims day-zero support for, is unstated.
630 downloads. The attention is well ahead of the use. Six days in, very few people have actually run this, so the claims are the publisher’s and nobody independent has reproduced the on-device figure.