Small open models got good enough this month that running one on your own machine is a real option, not a weekend project. A local model costs nothing per token, keeps your code and prompts on your computer, and still works when an API is down or a model gets retired. This guide covers the four tools most people use to run them (Ollama, LM Studio, llama.cpp and MLX), three open models from recent Hotfix Daily issues worth trying, how much memory you need, and how to plug a local model into a coding agent. Every version, price and command below comes from the maker's own page, checked on October 28, 2026.
Our take: start with Ollama if you live in a terminal, LM Studio if you'd rather click. Reach for llama.cpp when you want every knob, and MLX when you're on a Mac and want the fastest Apple-native path or plan to fine-tune.
New open models ship every week
Hotfix Daily tells you which ones are worth a download, with the source. Free, weekdays, about 3 minutes.
Join free and get 13 ready-to-paste Starter Prompts for coding agents, vibe coding and game dev right away.
All four run models on your own hardware. The difference is how much they do for you. Ollama and LM Studio pick a quantized build, download it and start a server. llama.cpp and MLX are the engines underneath: LM Studio says its runtime has "MLX and llama.cpp under the hood."
Install:curl -fsSL https://ollama.com/install.sh | sh on macOS or Linux, or irm https://ollama.com/install.ps1 | iex in Windows PowerShell. Then ollama run flint downloads and starts a model in one step.
What you get: a model library with ready-made builds (Flint is a 23 GB download with a 256K context), and a local server at http://localhost:11434 with an OpenAI-compatible API at /v1 (chat completions, responses, embeddings, tools and vision). To change the context size for OpenAI-style clients, set PARAMETER num_ctx in a Modelfile. The latest release, v0.40.12, added Liquid AI's LFM3 family, though its notes say LFM3 runs only on Apple hardware in that release, with Linux and Windows "coming soon."
Cost: running models on your own hardware "is always unlimited." Ollama also sells cloud access to bigger open models: pay as you go from $0, Pro at $20/month ($200/year), Max at $100/month and Team at $500/month. A cloud model's name ends in :cloud, so you always know which one you're using.
Our take: the shortest path from zero to a working local model and an API your other tools can call. Just watch which model names end in :cloud if you want everything to stay on your machine.
LM StudioElement Labs · desktop app, CLI, headless server
Install: download LM Studio 0.4.26 from the download page. For servers and CI there's llmster, a headless daemon (curl -fsSL https://lmstudio.ai/install.sh | bash). The lms CLI handles lms get (download), lms load, lms ls, lms chat and lms server start.
What you get: search, download and chat with GGUF and MLX models, then serve them on a local server with OpenAI-compatible and Anthropic-compatible endpoints (the docs' example is lms server start --port 1234). LM Studio now also ships Bionic, an agent app for open models. Bionic 1.1.9 (Oct 21, 2026) is the newest version in the changelog, and it can use LM Studio for local inference when both are running.
Cost: the Free plan ($0) includes the Bionic agent and running local models with llama.cpp and MLX. Bionic+ ($20/month) and Pro ($100/month) add US-hosted open models in the cloud.
Our take: the friendliest way to try a model before you commit to it, and the easiest way to see which quantization fits your memory. Its server makes it a drop-in backend for agents that speak the OpenAI or Anthropic API.
llama.cppggml-org · open source (MIT) · CLI, server, library
Install:curl -LsSf https://llama.app/install.sh | sh (or irm https://llama.app/install.ps1 | iex on Windows), Docker, prebuilt binaries from the releases page, or build from source. Then llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF downloads a model from Hugging Face and chats, and llama serve -hf … starts an OpenAI-compatible server with a web UI.
What you get: plain C/C++ with no dependencies, 1.5-bit to 8-bit quantization, Apple silicon as "a first-class citizen," CUDA, AMD (HIP), Vulkan and SYCL backends, and CPU+GPU hybrid inference for models bigger than your VRAM. Release v0.8.0 (Oct 28, 2026) added the LFM3 family and Flint, among other new architectures. The server README lists the default port as 9931 (change it with --port).
Our take: the engine new models land in first, and the one to use when an app hides a setting you need. If a model ships a GGUF file, it runs here.
MLX and mlx-lmApple (ml-explore) · open source (MIT) · Python
Install:pip install mlx-lm (or conda install -c conda-forge mlx-lm). Then mlx_lm.generate --model <model> --prompt "…", mlx_lm.chat for a chat session, and mlx_lm.server to serve it.
What you get: MLX is "an array framework for machine learning on Apple silicon," and mlx-lm runs and fine-tunes LLMs with it. mlx_lm.convert --model … -q makes your own quantized copy. MLX itself also installs on Linux with a CUDA backend (pip install mlx[cuda]) or CPU-only (pip install mlx[cpu]). Many model makers now publish MLX builds directly: Liquid AI has 4-, 5-, 6- and 8-bit MLX versions of LFM3-1.7B on Hugging Face.
Our take: on a Mac, the most direct route to Apple-native speed, and the easiest place to try a small fine-tune. Not the first stop on Windows.
2. Small open models worth trying (from recent issues)
Three open-weight models we covered this month, with what their makers and model pages say today. Benchmark numbers are the publisher's own, so treat them as a starting point and test on your own tasks.
Recursive Flint40B mixture-of-experts · about 2.5B active · Apache 2.0
Built for local coding agents. Recursive reports more than 125 tokens/second on an M4 Pro MacBook Pro at short contexts, up to 512K context (256K recommended), and 74.2% on SWE-bench Verified. It works with llama.cpp, Ollama, LM Studio, Unsloth, vLLM and SGLang. On Ollama it's ollama run flint, a 23 GB download. Recursive's own advice: break work into subtasks, because it may need more guidance than larger closed models.
Our take: the one to try first if you have a Mac or GPU with 32 GB or more and want a local coding agent. Covered in issue #2.
Liquid AI LFM3about 2B · six models · LFM Open License v1.0
Released Oct 27, 2026: text, vision, audio, omni and two decision models on one 1.7B backbone. The text model, LFM3-1.7B, fits in about 1 GB at 4-bit, has a 256K context, and needs 1.6 GB of KV cache for a full 128K context. Liquid says it beats Qwen3.5-9B on WebArena-Verified (38.42 vs 28.94) and AutomationBench (41.29 vs 15.10), and that bigger models still win on agentic coding and knowledge. Official GGUF and MLX builds are on Hugging Face, and Ollama has ollama run lfm3 (Apple hardware only for now). The license is not Apache: commercial use is licensed only for organizations under $10 million in annual revenue.
Our take: a background helper that runs on almost any laptop, good for tool calls, routing and triage. Not your refactoring model. Covered in issue #6.
A dense coding-agent model. On its model card it scores 65.5 on DeepSWE vs 42.5 for Qwen3.8-27B, while Qwen edges it on SWE-Bench Pro (61.7 vs 60.3). The official weights are BF16, about 66 GB on Hugging Face, and the card's instructions cover Transformers, SGLang and vLLM rather than Ollama or llama.cpp. Community GGUF and MLX conversions exist on Hugging Face, but they don't come from Nanbeige. For agent and tool use, the card recommends preserve_thinking=True and tool_call_format="xml".
Our take: a workstation model, not a laptop one. Worth it if you already have a big GPU and want a stronger local agent than the small models. Covered in issue #4.
A model has to fit in memory (VRAM on a graphics card, or unified memory on Apple silicon) with room left over for its context. The download size is a good first guess for the weights:
About 1 GB: LFM3-1.7B at 4-bit, per Liquid AI. Runs on almost anything, including older laptops.
About 23 GB: Flint on Ollama. Plan on a 32 GB Mac or a GPU setup with similar memory.
About 66 GB: Nanbeige4.3-33B at full BF16. Quantized community builds are smaller, at some cost in quality.
Context costs memory too. Long contexts need extra room for the KV cache (1.6 GB for LFM3 at 128K). If a model is slow or crashes, lower the context before you blame the model.
Too big still runs, slowly. llama.cpp can split a model between GPU and CPU, but each layer that spills onto the CPU costs speed.
Our take: rough math for a 4-bit build is about half a gigabyte per billion parameters, plus context. That's our rule of thumb, not a vendor number, so check the actual file size before you download.
4. Use a local model with a coding agent
Every tool above can run a local server that speaks an API your agent already knows. Point the agent's base URL at it and pick the local model:
Ollama:http://localhost:11434/v1 for OpenAI-style clients (the API key is "required but ignored"). Model pages list one-line launchers, for example ollama launch claude --model flint or ollama launch opencode --model flint.
LM Studio:lms server start, then use its OpenAI- or Anthropic-compatible endpoints.
llama.cpp:llama serve -hf <repo> for an OpenAI-compatible server.
MLX:mlx_lm.server.
A local model takes your API bill out of the picture, but not the agent's risks: it still runs commands as you. Keep approvals and sandboxing on, as in our guide to running coding agents safely. Want an agent that's built around bring-your-own models? See the free and open source picks in Which AI coding agent should you use?
Our take: pair a local model with small, well-scoped tasks: write the test, explain the stack trace, rename across a file. Hand the multi-file refactors to a bigger model.
Quick-start checklist
Check your free memory (RAM, or VRAM on a GPU) before picking a model.
Install Ollama or LM Studio and run a small model first, like LFM3-1.7B, to make sure everything works.
Move up to Flint if you have the memory and want a coding agent.
Read the license on the model page before shipping it in a product. Apache 2.0 and the LFM Open License are not the same.
Start the local server and point your agent at it.
Keep the agent's approvals and sandbox on.
FAQ
What is the easiest way to run an AI model locally?
Install Ollama or LM Studio. Ollama is a command line tool and local server: install it, then run a model with one command such as ollama run flint. LM Studio is a desktop app where you search for a model, download it and chat, and it can also serve the model to other apps. Both are free for local use as of Oct 28, 2026.
Do I need a GPU to run AI models locally?
Not always. llama.cpp runs on plain CPUs and splits work between CPU and GPU when a model does not fit in video memory, and Apple silicon Macs use shared memory for the GPU. The limit is usually memory: a model has to fit in RAM or VRAM, and it gets slow when it does not. Small models like LFM3-1.7B fit in about 1 GB at 4-bit; Flint is a 23 GB download on Ollama.
Which small open models are worth trying for coding agents?
From recent Hotfix Daily issues: Recursive Flint (40B mixture-of-experts, about 2.5B active, Apache 2.0) for local coding agents, Nanbeige4.3-33B (dense 33B, Apache 2.0, 512K context) if you have a big GPU, and Liquid AI LFM3-1.7B for tool calls and triage on almost any laptop. Their makers say bigger models still win on hard coding work.
Is running a model locally really free?
The software is: Ollama, llama.cpp and MLX are MIT-licensed, and LM Studio has a $0 plan for local models. You pay in hardware, power and time. Ollama and LM Studio also sell optional cloud plans (Ollama Pro $20 a month, LM Studio Bionic+ $20 a month) for models too big to run at home, but you do not need them to run models locally.
Can I use a local model with Claude Code or other coding agents?
Yes, through a local server. Ollama serves an OpenAI-compatible API at http://localhost:11434/v1 and its model pages list launch commands such as ollama launch claude --model flint. LM Studio offers OpenAI- and Anthropic-compatible endpoints from its local server, and llama.cpp has llama serve. Point the agent at that URL and pick the local model.
Found this useful? Hotfix Daily covers new open models and tools every weekday. Get the updates free, plus 13 Starter Prompts. Or share it on X.
How we write these: we read each tool's and model's official pages on the date shown and link the source for every version, price and command we name. Benchmark scores are the publisher's own. "Our take" lines are our opinion. Tools change often, so check the linked page before you rely on a detail. This guide has no affiliate links.
Get the next issue in your inbox
Five things worth your attention in AI, agents, vibe coding and game dev, every weekday. Plus 13 free Starter Prompts and the AI Builder Starter Stack PDF when you join.
