💻

Model Parameter Memory Calculator (fp16 / int8)

See how much memory a model’s weights need at each precision — fp32, fp16, int8 or int4 — plus runtime overhead, from just the parameter count.

Parameters

Precision

Runtime overhead

Loading calculator…

How it works

Weights (GB) = parameters (billions) × bytes per parameter · Total = weights × (1 + runtime overhead%)

Weight memory follows a clean rule: parameters in billions times bytes per parameter gives gigabytes, so a 7B model is 14 GB in fp16, 7 GB in int8 and 3.5 GB in int4. Quantization is what lets big models fit on smaller GPUs, trading a little accuracy for a large memory cut. On top of the weights you need room for the KV cache and activations at inference — the overhead term — which grows with context length and concurrency, so pick a precision that leaves comfortable headroom rather than one that just barely fits.

Worked example

A 7B model in fp16 (2 bytes) needs 7 × 2 = 14 GB for weights. Adding 20% runtime overhead gives about 16.8 GB — meaning a 24 GB GPU fits it comfortably, while a 16 GB card would need int8 (7 GB weights) instead.

Frequently asked questions

Why does parameters × bytes give gigabytes directly?

Because a billion (1e9) parameters times the bytes each takes, divided by a billion bytes per GB, cancels out — so 7 billion params × 2 bytes = 14 GB with no extra arithmetic.

Is int4 always worth it?

Int4 halves memory again versus int8 and often keeps most of the quality on large models, but small models and precision-sensitive tasks can degrade noticeably. Benchmark on your task; use int4 when memory is the binding constraint and a small quality hit is acceptable.

Why add runtime overhead?

Serving a model also needs memory for the KV cache (which grows with context length and batch size) and activations. Twenty percent is a rough default for short contexts; long-context or high-concurrency serving can need far more, so raise it accordingly.

Related calculators

GPU VRAM for Model Calculator

Estimate the GPU memory needed to load an AI model from its parameter count and numeric precision.

Batch Size Memory Calculator

Estimate the activation memory a training batch consumes from batch size, sequence length, hidden size and layer count — the term that scales with batch and blows up GPU memory.

Model Training Cost Calculator (GPU-Hours)

Turn a training run — number of GPUs, wall-clock hours and price per GPU-hour — into total GPU-hours and dollar cost, the way cloud accelerator bills are actually rated.

Embedding Storage Size Calculator

Work out how much disk or RAM a set of embeddings needs from the vector count, dimensionality and numeric precision, including index overhead.

Educational estimate — real costs and memory depend on your provider, framework, model architecture and pricing, which change often. Verify against your own bills and profiling before committing budget.