Tuning a thread pool

Updated: October 26, 2022

By tuning parameters such as lo_water and hi_water, you can create thread pools that handle variable loads while striking a good balance of CPU and memory usage.

A dynamic thread pool (i.e. a pool in which worker threads are created and destroyed as needed) can help a server manage variable loads. Problem is, the pool creates new threads—which contribute to overhead—at the worst time possible: when the server is getting hit by a heavy load. This approach is preferable to not creating the threads, but comes at a cost nonetheless.

In some cases, you can avoid the issue by setting the pool to a fixed size. To take this approach, you need to:

Of course, setting the pool to a fixed size isn't always appropriate. For instance, you may need to tune your parameters to handle the following dynamic scenarios:

A further concern is the thrashing that can occur when load bursts cause threads to be repeatedly created and then destroyed. To prevent this issue, increase the difference between lo_water and hi_water.