Thread Pool Size for a Workload
Problem
A service handles requests on a fixed thread pool. Each request computes on the CPU for compute_ms and then waits on the network or a disk for wait_ms, and while a thread waits its core can run another thread. A common sizing rule is threads = cores × target utilization × (1 + wait / compute). Given the core count, the target utilization as a whole percentage, and the two times in milliseconds, return the pool size rounded up. Use integer arithmetic so the answer does not depend on floating-point rounding.
Examples
Input: cores = 8, utilization = 100, wait_ms = 90, compute_ms = 10
Output: 80
Why: each core is busy only a tenth of the time, so ten threads share itInput: cores = 4, utilization = 50, wait_ms = 30, compute_ms = 20
Output: 5
Why: 4 × 0.5 × 2.5 = 5Input: cores = 16, utilization = 100, wait_ms = 0, compute_ms = 5
Output: 16
Why: edge case, pure CPU work gains nothing from more threads than coresHints
0 / 3
Ask what fraction of its life a thread actually spends on a core. The rest of the time the core is free for someone else.
1 + wait / compute is the same as (compute + wait) / compute, so the whole formula is one fraction with an integer numerator and denominator.
Multiply cores, the percentage and (compute + wait) for the numerator, use 100 × compute as the denominator, and take the integer ceiling with negated floor division.
Solution
A thread holds a core only for the compute part of each request, so one core can keep (compute + wait) / compute threads busy, and the utilization target scales that down to leave headroom. Writing the rule as a single fraction keeps every value an integer until the very end, and the ceiling is taken with negated floor division, -(-a // b), which rounds up exactly. Rounding up rather than down matters: a pool one thread short leaves a core idle during every wait. Any positive fraction rounds up to at least one thread. Time and space are O(1).
def pool_size(cores, utilization_pct, wait_ms, compute_ms):
# threads = cores * utilization * (1 + wait / compute), rounded up
num = cores * utilization_pct * (compute_ms + wait_ms)
den = 100 * compute_ms
return -(-num // den) # integer ceiling
print(pool_size(8, 100, 90, 10)) # -> 80
print(pool_size(4, 50, 30, 20)) # -> 5
print(pool_size(16, 100, 0, 5)) # -> 16
print(pool_size(1, 10, 5, 10)) # -> 1Stuck on the idea rather than the code? Sizing a Thread Pool covers it.