Softmax Calculator
Softmax of a list of logits with optional temperature — the probability of each class, the top class, log-sum-exp and the entropy of the distribution.
How it works
softmax(z)ᵢ = e^(zᵢ/T) ÷ Σⱼ e^(zⱼ/T), computed as e^(zᵢ/T − max) for numerical stability; log-sum-exp = max + ln Σ e^(zⱼ/T − max); H = −Σ pᵢ log₂ pᵢ
Softmax turns a vector of raw scores (logits) from a neural network into a probability distribution: every output is positive and they add up to 1, and larger logits get exponentially more weight. Dividing the logits by a temperature T is how language models control randomness — T < 1 makes the top class dominate, T > 1 spreads probability more evenly. Subtracting the maximum logit first gives identical results without overflow.
Worked example
Logits 2.0, 1.0 and 0.1 at T = 1 give probabilities 65.90%, 24.24% and 9.86%; class 1 is the top class with 65.90%, the log-sum-exp is 2.4170 and the entropy is 1.2216 bits.
Sources: Goodfellow, Bengio & Courville, Deep Learning (MIT Press), §6.2.2.3 Softmax units · Softmax function — Wikipedia
Frequently asked questions
What does temperature do?
It divides every logit before the exponent. Low temperature (e.g. 0.2) pushes the distribution towards the single largest logit; high temperature (e.g. 2) pushes it towards uniform. T = 1 is the plain softmax.
Why subtract the maximum logit?
e^1000 overflows a computer’s floating-point range. Subtracting the largest logit leaves the result unchanged mathematically but keeps every exponent ≤ 0.
Softmax or sigmoid?
Sigmoid gives independent probabilities for each label (multi-label problems); softmax gives mutually exclusive class probabilities that sum to 1.
What does the entropy tell me?
How uncertain the distribution is: 0 bits means one class has all the probability; log₂(number of classes) means all classes are equally likely.
Related calculators
This calculator is for educational purposes. Double-check important results before acting on them.