Creative Hardware
A 30-Million-Parameter Model Trained From Scratch, Running Offline on an ESP32 — With the Dataset Licences Published
PocketGPT is a 30,460,864-parameter decoder-only transformer, trained from random initialisation on roughly 300 million token positions, running offline on an ESP32-S3 handheld with a round touchscreen. It generates 0.742 tokens per second on a XIAO. The README says bluntly that answers can be wrong, repetitive or unrelated. It also ships a dataset-by-dataset licence table with pinned git revisions and an explicit list of what could not be verified.