Thread Pool Size Calculator

How it works

Little's law: concurrency L = arrival rate λ × service time W · Threads = ⌈L ÷ target utilisation⌉

Little's law says the average number of requests being processed at once equals the arrival rate times how long each takes — so that product is the raw number of workers you need to keep up. But running a pool at 100% utilisation leaves no slack for bursts and makes queue time explode, so you divide by a target utilisation (say 80%) to add headroom and round up to a whole thread. For CPU-bound work the pool should also not far exceed the core count; for I/O-bound work the utilisation-adjusted figure is the guide.

Worked example

At 500 req/s with 20 ms average service time, concurrency = 500 × 0.020 = 10 requests in flight. Targeting 80% utilisation, size the pool at 10 ÷ 0.80 = 12.5, rounded up to 13 threads. Those 13 threads can process up to 13 ÷ 0.020 = 650 req/s at full tilt, the headroom above the 500 req/s load.

Frequently asked questions