The best local AI model for your computer depends mainly on your RAM. With 8GB, run a small 3B model like Gemma 2B or Phi-3 Mini. With 16GB, an 8B model like Llama 3 8B or Mistral 7B is the sweet spot. With 32GB or a strong GPU, you can run 13B to 70B models for near-cloud quality. Always use quantised (Q4) versions to fit larger models into less memory. Match the model size to your RAM and everything runs smoothly and free.
Once you have a tool like Ollama or LM Studio installed, the next question is which model to actually run — and the honest answer is that it comes down to your computer’s memory more than anything else. Pick a model too big and it crawls or won’t load; pick one well-matched to your hardware and you get a fast, capable, private assistant for free. This guide maps the best local AI models to each common amount of RAM, so you can choose confidently rather than guessing.
Why RAM is the deciding factor
AI models are loaded into memory to run, so the amount of RAM (or VRAM on a dedicated graphics card) you have sets a hard ceiling on the size of model you can use. Model sizes are measured in billions of parameters — 3B, 7B, 13B, and so on — and bigger generally means smarter but hungrier for memory. A model that needs more RAM than you have either refuses to load or spills over and runs painfully slowly, which is why matching size to memory matters more than any other single choice.
The clever trick that widens your options is quantisation, a form of compression that shrinks a model’s memory footprint with only a small loss in quality. A quantised four-bit version of a model — usually labelled Q4 — can use roughly half or a third of the memory of the full version while performing almost as well, which is why Q4 models are the popular default. Throughout this guide, assume you are downloading Q4 versions, because they let you run a noticeably more capable model on the same hardware.
8GB RAM: small but capable
If your computer has 8GB of RAM and no dedicated graphics card, you are in small-model territory, and that is perfectly fine for everyday use. Models around three billion parameters — Google’s Gemma 2B, Microsoft’s Phi-3 Mini, or a small Llama variant — load comfortably and handle writing, summarising, answering questions, and light coding well. They won’t tackle the most complex reasoning, but for the tasks most people actually use AI for, a well-chosen 3B model on 8GB is genuinely useful and pleasantly quick.
The key on limited memory is to keep expectations realistic and close other heavy programs while the model runs, since your browser and other apps compete for the same RAM. Stick to one small model, use a Q4 version, and avoid trying to force a 7B model that will only frustrate you. Within those limits, an 8GB machine makes a fine, free, offline assistant.
16GB RAM: the sweet spot
Sixteen gigabytes of RAM is where local AI really opens up, comfortably running the seven-to-eight-billion-parameter models that most enthusiasts consider the best balance of capability and speed. Llama 3 8B and Mistral 7B are the standout all-rounders here, strong at writing, reasoning, and general knowledge, while Qwen and Gemma models in the same range are excellent alternatives worth trying. For the majority of laptops and desktops sold today, this is the tier that delivers a genuinely satisfying local AI experience.
At 16GB you can also keep a couple of models installed and switch between them — perhaps a fast 7B for quick chat and a slightly larger or specialised one for particular jobs like coding. Responses are quick, quality is high enough for real work, and everything still runs privately and free. If you are buying or specifying a machine with local AI in mind, 16GB is the amount to aim for as a practical minimum.
| Your RAM | Best model size | Examples |
|---|---|---|
| 8GB | 3B (Q4) | Gemma 2B, Phi-3 Mini |
| 16GB | 7–8B (Q4) | Llama 3 8B, Mistral 7B |
| 32GB | 13–14B (Q4) | Larger Llama/Qwen variants |
| Strong GPU / 64GB | 70B (Q4) | Llama 3 70B |
32GB and beyond: near-cloud quality
With 32GB of RAM or a capable dedicated graphics card, you move into the larger models — thirteen to fourteen billion parameters and, at the top end, seventy-billion-parameter models — that start to approach the quality of cloud services for many tasks. These bigger models reason more reliably, follow complex instructions better, and produce more polished writing, at the cost of needing serious memory and running a little slower than the small ones. For power users who want the best local experience, this is the tier that delivers it.
A dedicated GPU with plenty of VRAM changes the equation further, because models run far faster on a graphics card than on a processor alone. If you have a strong gaming or workstation GPU, you can run large models at genuinely quick speeds, which is why enthusiasts building machines for local AI prioritise VRAM. Most people don’t need this tier, but if you have the hardware, it turns local AI from ‘good enough’ into something that rivals paid cloud tools for a great many jobs.
If you have a dedicated graphics card, its VRAM is what matters most for speed. A model that fits entirely in VRAM runs dramatically faster than the same model running on your processor and system RAM, so check your GPU’s memory when choosing a model size.
How to pick and test a model
The practical approach is to start one tier below what you think you can handle, confirm it runs smoothly, then experiment upward. Download a Q4 model matched to your RAM from the tables above, run a few of your real tasks through it, and judge both the speed and the quality of the answers. If responses are slow, step down a size; if they are fast and you want more depth, step up. Because everything is free, this experimentation costs nothing but a little download time and disk space.
It is also worth trying a few different model families at the same size, because Llama, Mistral, Qwen, and Gemma each have slightly different strengths and personalities. One may suit your writing style better, another may be stronger at code. Keeping two or three favourites installed and switching based on the task gives you the best of the local-AI world, all tuned to your specific machine and needs.
RAM determines whether a model runs well, but each model also occupies several gigabytes of storage. It is easy to download many while experimenting, so delete the ones you don’t keep to avoid quietly filling your drive.
FAQ
What model can I run with 8GB of RAM?
A small 3B model such as Gemma 2B or Phi-3 Mini, in a quantised Q4 version. These handle everyday writing and questions well; avoid larger 7B models, which will run slowly or fail to load on 8GB.
Is 16GB enough for local AI?
Yes — 16GB is the sweet spot, comfortably running excellent 7–8B models like Llama 3 8B and Mistral 7B. It is the practical minimum to aim for if you want a genuinely capable local assistant.
What does Q4 mean?
Q4 refers to four-bit quantisation, a compression that shrinks a model’s memory use with only a small quality loss. Q4 versions let you run a more capable model on less RAM, which is why they are the recommended default.
Do I need a graphics card?
No, but one helps enormously with speed. Models run on your processor and RAM without a GPU, just more slowly. A dedicated card with ample VRAM makes larger models fast and is what enthusiasts prioritise.
Getting the best speed from your model
Beyond matching model size to memory, a couple of tweaks squeeze out more speed. Closing other heavy programs frees RAM for the model, and on a machine with a dedicated graphics card, making sure the model runs on the GPU rather than the processor can multiply its speed. Both Ollama and LM Studio handle this largely automatically, but confirming your GPU is being used is worth the check on a capable machine.
If a model still feels sluggish, a more heavily quantised version — a smaller Q number — trades a little quality for a meaningful speed boost, which is often the right call for everyday chat. The goal is a model that responds quickly enough to feel conversational; a slightly smaller or more compressed model that answers instantly usually beats a larger one that makes you wait, for the kind of tasks most people run locally.
The Bottom Line
Choosing a local AI model is mostly about matching it to your memory: a 3B model for 8GB, a 7–8B model for the 16GB sweet spot, and 13B to 70B models for 32GB or a strong GPU — always in quantised Q4 form. Start one tier below your limit, confirm it runs smoothly, and experiment upward, trying a few model families to find your favourite. Get the match right and you have a fast, private, capable assistant that runs entirely on your own hardware, completely free.