Four weeks ago we wrote about generative models running on bare-metal microcontrollers and said the interesting number was where the floor is. PocketGPT is a different kind of entry in the same territory, and the difference is the part worth reading: nobody handed this one a pretrained model.
The maker credited as Marina — mmm1712 on GitHub, marinmandarinegirl on Hackster — trained a 30,460,864-parameter language model from random initialisation and got it running offline on an ESP32-S3, with a touchscreen chat keyboard. The project video went up 13 October 2026; the repository was last pushed 23 October, is MIT licensed, and has 88 stars.
The model
| Independent trained parameters | 30,460,864 |
| Architecture | Decoder-only transformer, 12 layers, hidden size 448 |
| Attention | 8 query heads, 2 key/value heads (grouped-query) |
| Tokeniser | SentencePiece, 6,144 tokens |
| Runtime limits | 64-token context, up to 24 generated tokens per reply |
| Measured speed | DevKit ~1.41 tokens/s; XIAO ~0.742 tokens/s |
Two design choices stand out as deliberate adaptations to the target rather than scaled-down defaults. The 6,144-token vocabulary is tiny — a tenth of a typical one — and that matters enormously here, because the embedding and output layers are where a small model spends its parameters. And grouped-query attention at 8:2 cuts the KV cache by four, which on a device with megabytes rather than gigabytes is the difference between running and not.
The 64-token context is the honest constraint. This is not a model that holds a conversation; it answers one short thing.
The hardware, in two variants
- XIAO ESP32-S3 with 8 MB flash and 8 MB PSRAM, a Seeed Round Display and a microSD card: the handheld, with an on-screen touch keyboard. Weights are split across flash, PSRAM-loaded data and the SD card — 21,717,088 bytes in total, with cache and tokeniser files separate.
- ESP32-S3 DevKit with 32 MB octal flash and 16 MB PSRAM: chat over USB serial, using a 20,241,616-byte deduplicated pack.
The two boards use different mixed-precision layouts and therefore different model packages, which the repository ships as release assets. Weights and tokeniser are published — no training run and no API key needed to reproduce the device.
What it can actually do, in the author’s own words
The README does not oversell, and the restraint is the most persuasive thing in it:
This is a personal experiment with limited capabilities. It can produce short greetings and basic conversational replies, but answers can be wrong, repetitive or unrelated. Even basic arithmetic is unreliable.
Then it prints recorded hardware outputs, with the timings:
| Prompt | Response | Board | Time |
|---|---|---|---|
hello | Hello. | XIAO | ~6.4 s |
how are you? | I'm doing okay. | XIAO | ~11.8 s |
say something nice | You're doing better than you think. | DevKit | — |
Eleven seconds for four words. The raw serial logs for both boards are committed to the repository, with dated directories.
The project also notes that its serial test “checks for a nonempty reply, a positive token count and no reported inference errors. Passing confirms generation completed, not that the answer is factually correct.” That sentence is a model of what a test claim should sound like.
The provenance table is the real find
Here is what makes this project worth writing about rather than merely admiring.
docs/DATA_PROVENANCE.md, reviewed 13 October 2026, is a dataset-by-dataset inventory of the training sources with licence labels and pinned commit revisions:
| Source | Revision | Licence label |
|---|---|---|
| TinyStories | not pinned | CDLA-Sharing-1.0 (current card) |
| FineWeb-Edu | not pinned | ODC-By-1.0; card also references Common Crawl terms |
| SODA | fdc848ab… | CC-BY-4.0 |
| OASST1 | fdf72ae0… | Apache-2.0 |
| UltraChat 200k | c02ed026… | MIT |
| smol-smoltalk | f80219b4… | Apache-2.0 |
And then it says what it cannot prove. On the two unpinned sources: “exact upstream snapshot still missing” and “Do not substitute a revision from another experiment.” On the tokeniser, which has a separate provenance chain: a corpus-builder script targets Dolly-15k and GSM8K, but “a matching execution log or frozen source manifest for this tokenizer was not found,” so “Dolly and GSM8K remain possible sources, not confirmed inputs.” On the whole thing: “Current card labels are not proof that unpinned historical downloads had identical terms.”
Read those sentences again with this week’s other releases in mind. Getty Images is in Chapter 11 still selling “permissioned content.” Bria’s FIBO 2.0 advertises 100% licensed data and ships non-commercial gated weights. Gradium’s Phonon is “Apache-2.0” for one of seven models. Canva can at least name its opt-in programme.
The most rigorous training-data disclosure of the week came from a solo maker’s 30-million-parameter toy that takes eleven seconds to say it is doing okay.
What to take from it
If you want a chatbot, this is not it, and the author says so first. If you want to learn how a transformer actually fits into constrained memory, the repository is an unusually complete worked example: the training code and settings are included, the deployment layouts are documented per board, and docs/MODEL_CARD.md covers the quantisation.
And if you ship anything trained on public datasets, copy the provenance file. It took one person one day. It is the single cheapest credibility improvement available to anybody releasing a model.
Sources
- mmm1712/Local_AI_esp32 — PocketGPT on ESP32-S3 (MIT)
- Model and tokenizer provenance review
- Model asset releases (v0.1.0)
- I Trained My Own 30M Parameter AI to Run on an ESP32 — Hackster
- Project video