Shannon Entropy Calculator
Information entropy of a distribution from counts or probabilities — bits, nats, perplexity, maximum possible entropy and normalised entropy (evenness).
How it works
H = −Σ pᵢ·log₂ pᵢ (bits); H_max = log₂ k; perplexity = 2ᴴ; evenness = H ÷ H_max
Shannon entropy measures the average information — or uncertainty — in one draw from a distribution. A certain outcome has 0 bits; a fair coin 1 bit; a fair six-sided die 2.58 bits. Perplexity turns entropy back into an “effective number of equally likely outcomes”, which is how language models are scored. Normalised entropy is the evenness measure used in ecology and in decision-tree splitting (information gain).
Worked example
Probabilities 0.5, 0.25, 0.15, 0.10: H = 1.74 bits against a maximum of 2 bits — evenness 0.87, perplexity 3.34.
Frequently asked questions
Bits, nats or dits?
Only the logarithm base changes: bits use log₂, nats logₑ, dits log₁₀. Multiply bits by 0.693 for nats.
How is this related to diversity indices?
The Shannon diversity index in ecology is the same formula in nats; exp(H) is the effective number of species.
Related calculators
This calculator is for educational purposes. Double-check important results before acting on them.