Tuning a thread pool
By tuning parameters such as lo_water and hi_water, you can create thread pools that handle variable loads while striking a good balance of CPU and memory usage.
A dynamic thread pool (i.e. a pool in which worker threads are created and destroyed as needed) can help a server manage variable loads. Problem is, the pool creates new threads—which contribute to overhead—at the worst time possible: when the server is getting hit by a heavy load. This approach is preferable to not creating the threads, but comes at a cost nonetheless.
In some cases, you can avoid the issue by setting the pool to a fixed size. To take this approach, you need to:
- Determine, whether through testing or system analysis, the number of threads that the thread pool requires.
- Set maximum to the number of required threads.
- Set both lo_water and hi_water to the same number as maximum.
- A high initial load that drops down to a lower, steady load — In this case, you likely need to set hi_water to be significantly lower than maximum. This configuration allows many threads to be created at the high load point, but will recover the memory overhead of the per-thread data as the load decreases.
- An ongoing series of burst loads — In this case, you would set hi_water to maximum. This configuration assumes that, once you've hit the load required to create a new thread, you're likely to hit that load again. It thus avoids the overhead of thread creation each time the load occurs. Compared to an approach that creates and destroys threads, this configuration commits more memory, but reduces CPU overhead and latency for future requests to the pool.
A further concern is the thrashing that can occur when load bursts cause threads to be repeatedly created and then destroyed. To prevent this issue, increase the difference between lo_water and hi_water.