GPU VRAM for Model Calculator
Estimate the GPU memory needed to load an AI model from its parameter count and numeric precision.
How it works
Weights (GB) = parameters (billions) × bytes per parameter · Total VRAM = weights × (1 + overhead%)
A model's weights occupy parameters × bytes-per-parameter of memory, and because a billion parameters times one byte is almost exactly one gigabyte, the weights in GB are just the parameter count in billions times the bytes per parameter. Precision is the biggest lever: FP16 halves FP32, and INT8 or INT4 quantization halve it again each step, at some cost to accuracy. On top of the weights you need headroom for activations and the attention KV cache, which grows with context length — the overhead percentage captures that.
Worked example
A 7-billion-parameter model in FP16 uses 7 × 2 = 14 GB for weights. Add 20% for activations and KV cache and you need about 16.8 GB — just beyond a 16 GB card, but comfortable on 24 GB. Quantized to INT4 (0.5 byte) it needs only 7 × 0.5 × 1.2 ≈ 4.2 GB.
Frequently asked questions
Why about 1 GB per billion parameters at 1 byte?
One billion parameters × 1 byte = 10⁹ bytes, and a gigabyte is 10⁹ bytes (or 2³⁰ ≈ 1.07 × 10⁹ for a gibibyte). The two are close enough that the parameter count in billions is a handy stand-in for gigabytes at one byte each.
Does context length change the VRAM?
Yes. Longer contexts enlarge the attention KV cache, which can dominate memory for long prompts. Raise the overhead percentage for large context windows, or add the KV cache separately for a precise figure.
What does quantization cost?
Dropping from FP16 to INT8 or INT4 roughly halves memory at each step and speeds inference, but introduces small accuracy losses. Modern 4-bit methods keep quality close to the original for many tasks, which is why they are popular for running large models on consumer cards.
Related calculators
This calculator is for educational purposes. Double-check important results before acting on them.