🖥️ Can My Computer Run This AI Model?

How the estimate works

Each parameter takes its number of bits divided by eight in bytes, so a billion parameters at 8-bit is about 1 GB, and at 4-bit about 0.5 GB. On top of the weights, the model needs room for the conversation it is working on (the context) and for the software running it, so the tool adds about 20%. Long contexts need more than that.

Full speed, slow, or not at all

If the whole model fits in the graphics card's memory, it runs at full speed. If it only fits when part of it sits in system memory, it still runs, but much slower, because the graphics card has to wait for data from RAM. The tool counts only three quarters of your RAM, because the operating system and other programs need the rest. Apple Silicon Macs share one pool of memory: enter it as RAM and set VRAM to 0.

Limits

This is a rule of thumb, not a guarantee. The exact need depends on the model's architecture, the runtime (llama.cpp, Ollama, LM Studio and others differ) and the context length you choose. Speed is not estimated.