Running an open-weight AI model locally — entirely on your own machine, with no API key, no per-token cost, and no dependency on a service staying online — has become genuinely accessible. Two tools cover most people’s needs: LM Studio for a point-and-click experience, and llama.cpp for command-line control and scripting. Meta’s newly-released Muse Glimmer, covered here today, is exactly the kind of model this workflow is built for.
Watch: How to Run Local LLMs with Llama.cpp: Complete Guide (pookie, YouTube)
Step 1: Pick the tool that matches your comfort level
LM Studio is the better starting point for most people: a free, cross-platform (Windows/Mac/Linux) app with a built-in model browser, a chat interface, and no command-line requirement at all — install it, pick a model, and start chatting within about half an hour. llama.cpp is the underlying engine LM Studio itself is often built on, and using it directly gives finer control (scripting, serving an OpenAI-compatible API for other tools to call) at the cost of working from a terminal. Start with LM Studio; move to llama.cpp specifically once you know why you need the extra control.
Step 2: Understand GGUF — the format that makes this all work
Most locally-run open models are distributed as GGUF files — a quantized format that shrinks a model’s memory footprint (trading a small amount of precision for a dramatically smaller file and lower RAM/VRAM requirement) so it can actually run on consumer hardware rather than requiring datacenter-grade GPUs. LM Studio’s built-in Hugging Face browser filters specifically for GGUF-compatible models, which is the practical reason nearly every fresh open-weight release (including community repacks like the ones covered here for MiniMax H3) gets a GGUF version within hours of launch — it’s the format that makes “run this locally” actually possible for most people.
Step 3: Match model size to your actual hardware
A model’s parameter count (7B, 13B, 30B, and so on) roughly determines how much RAM or VRAM it needs at a given quantization level — a 30B model like Muse Glimmer needs meaningfully more headroom than a 7B model, even in a compressed GGUF format. Both LM Studio and llama.cpp will generally tell you upfront whether a given model/quantization combination is likely to run acceptably on your specific hardware; pay attention to that guidance rather than downloading the largest model available and hoping.
Step 4: Confirm it’s actually running offline
Once a model loads and responds, disconnect from the internet and confirm it still works — this is the actual point of local inference, and it’s worth verifying directly rather than assuming. A model that requires a live connection to function isn’t running locally in the way that matters for privacy, cost, or reliability; genuine local inference should work identically with the network cable pulled.