You are coding, and a language model is already part of the process - generating a function, explaining a bug, cleaning up messy code. It feels excellent, right up until you hit the limits.
Free tokens vanish quickly. Premium subscriptions add up. Soon you are switching between accounts just to keep working, which breaks flow and gets tiring fast.
So what if you could run a capable model with no request limits, no account juggling, and no data leaving your machine?
That is what Ollama is for.
What Ollama actually is
Ollama is a tool that runs large language models - Llama, Mistral, Code Llama, Gemma, Qwen and many others - directly on your own computer.
Instead of sending your question to a company's servers, you download the model once and run it locally.
You ask a question → it travels to a remote server → you get a reply. Metered, network-dependent, and your prompt leaves your machine.
You download the model once → you ask it questions → it replies locally. No internet needed after setup, no per-request cost.
Why it is worth setting up
- Privacy - everything runs locally, so your code and prompts never leave your machine. This alone justifies it for proprietary or regulated work.
- Offline use - models keep working on a plane, on bad hotel wifi, or behind a restrictive corporate network.
- Flexibility - swap between open models depending on the task, rather than being locked to one provider.
- Developer friendly - it ships with a CLI and a local HTTP API, so you can build against it like any other service.
- No metering - once the model is downloaded, you can run it as often as your hardware allows.
Who it is for
Developers building features without depending on a cloud API or paying per token during development.
Privacy-conscious users who cannot send data to an external service for legal or contractual reasons.
Experimenters who want to compare open models without signing up for anything.
Getting started
Step 1 - Install
Download Ollama from ollama.com and install it like any other application. Builds are available for macOS, Windows and Linux.
Step 2 - Run a model from the terminal
ollama run llama3Reading that command:
ollama- the CLIrun llama3- pull (if needed) and start an interactive session with Llama 3
The first run downloads the model, which can be several gigabytes. Every run after that starts in seconds.
A few other commands worth knowing:
# download a model without starting a chat
ollama pull mistral
# list what is installed locally
ollama list
# remove a model you no longer need
ollama rm mistralStep 3 - Use the desktop app
Not a fan of the terminal? Ollama ships with a desktop application that gives you a normal chat interface over the same local models. Same engine, no command line.
Step 4 - Wire it into your editor
Most editors can point at a local Ollama instance:
- Install an extension that supports a custom or local model endpoint (Continue, Cody and several others do)
- Point it at
http://localhost:11434 - Use it for completion, explanation and refactoring without leaving your editor
Step 5 - Call it from your own code
Ollama exposes a local HTTP API, which makes it straightforward to build against:
curl http://localhost:11434/api/generate -d '{
"model": "llama3",
"prompt": "Explain what a React error boundary does, in two sentences.",
"stream": false
}'Is it free?
Ollama itself is free and open source, and the models it runs are open-weight models you download at no cost.
The real limit is your hardware. Larger models need more RAM and benefit enormously from a GPU. As a rough guide:
| Model size | Rough RAM needed | Realistic use |
|---|---|---|
| 3B parameters | ~4 GB | Quick completions, light tasks, older laptops |
| 7–8B parameters | ~8 GB | The practical sweet spot for most development work |
| 13B parameters | ~16 GB | Better reasoning, noticeably slower on CPU alone |
| 30B+ parameters | 32 GB or more | Needs a serious GPU to stay usable |
Begin with a 7B or 8B model. If it feels fast and useful, try something larger. If it feels sluggish, drop to a 3B model rather than concluding local models are not viable.
The short version
Ollama is a local model engine - the same capability you are used to, running on your own laptop, with you choosing the model and nothing leaving your machine.
For anyone working under privacy constraints, on unreliable connections, or simply tired of hitting request limits mid-task, it is one of the highest-value hours you can spend on your setup.
