AI & Creative Tools
A 35B Vision Model Running at 13 Tokens a Second in 3 GB on an iPhone
OpenBMB's MiniCPM-V 4.7 35B-A3B claims to be the first open-source MoE model that runs on-device without compromising performance: 13+ tokens/s in 3 GB of memory on an iPhone 17 Pro Max. Apache 2.0, with day-zero support for Rockchip, Houmo, Intel, AMD and Arm, and 302 likes on 630 downloads six days after release.