What is the ideal Thread Pool Size – Java Concurrency

We programmers do a lot of multi threading programming in professional life. Several times there comes a need when we required to write a Thread in java. Then, sometimes question arises in our mind that what should be the ideal thread pool size so that we can utilize the maximum CPU and increase the performance of java program.

Thread pool is a pool of worker threads. Worker thread is thread which can accept a task, then complete it and come back again to thread pool to accept another task.

We also come across to the concept of Java concurrency. Before moving ahead we should be aware of basic terminologies.

CPU Cores– It is a physical CPU capable of doing independent processing. In case of Hyper Threading, a core can do two or more tasks simultaneously. Its leads to logical processing.

Logical Processors– It is the processor seen by the operating system. But actually it does not exist physically.

Logical Processor = num. of core * num. of thread in each core = 4 * 2 =8

where No. of Core = 4 (ie. physical processor unit)

CPU Cores and Logical Processors

How to decide number of threads in a pool is very important to understand and if not, then it could spoil your program performance and memory management very badly.

What should be the size of thread pool

This concept applies to the fixed thread pool in which we have option to give number of threads and you should have many tasks to be executed.

Ideally, there is no fixed number which is ideal to be used in deciding the number of threads pool. It all depends on the use-case of the java program.

Therefore, there are two factors on which decision should be taken to decide the thread pool size.

1. Number of CPU Cores

A single CPU core will run one thread at a time. Lets, ignore the hyper threading for now.

A typical desktop generally is a quad core which means there are four cores in a CPU. While a server or cloud can have as many as cores in CPU.

So, if one core is there in CPU then we can run one thread at a time.

N cores = N Threads

It simply means more cores then more parallelism.

In case of Hyper Threading, considering the logical processors-

int poolSize  = Runtime.getRuntime().availableProcessors();

Sometimes it is also possible that a server or cloud has many cores but the application has only access to few cores. Then in that case, the available cores to the application will decide the number of thread pool size.

2. Type of tasks

There are two type of tasks:

  • CPU Bound
  • IO Bound

CPU Bound Tasks

Tasks which involve mathematical calculations and data marshaling etc. are known as CPU bound tasks.

Suppose there is one CPU core and one thread is running and has 2 task submitted. Then one task is submitted to thread one and once that completed then other task is submitted. In between the submission of two tasks there should not be any time gap to achieve the maximum utilization of CPU.

Maximum Utilization of CPU

In case, if there are two threads in one core CPU, then also there will be same amount of time will be taken by the CPU which it was taking in one thread. Therefore, making too many threads in one core CPU will not affect anything with respect to performance. Hence, in case of CPU bound tasks , number of threads should be equal to number of cores of CPU.

Multiple threads in CPU

IO Bound Tasks

Tasks which involve communication to other application through network calls like database, web services, micro services, outside caches etc.

IO Bound Tasks

In the above diagram, there is one thread and one task is submitted to it. When this task wait for IO operation to complete then CPU becomes free and inefficient. Once the IO call gives the response then it again starts working until the task is completed.

During the free time or wait time we can initiate one more thread and make it running so that maximum output of the CPU is gained and the thread one could be in wait state until the output is received from IO call.

Multiple tasks in IO operation

Therefore, in case of IO bound tasks having one core CPU we can run increase the number of threads and could gain the maximum utilization of CPU. Hence number of threads depends on the time it takes to complete the one task IO operation. This way we can apply same concept in case of multiple cores in CPU.

Here comes the formula to calculate the number of thread count for maximum utilization of CPU.

Ideal thread Count= Number of Cores * [ 1+ (wait time/CPU time)]

(wait time/CPU time) this is called the blocking co-efficient.

Other factors involved in deciding thread count

  • You should check whether other applications running on same CPU.
  • Are there any other threads running on same JVM.
  • Too many threads in same CPU may cause slow performance.

Conclusion

In this article I have explained what are the deciding factors for ideal thread pool size. Major factors depends on the type of tasks. There are two type of tasks i.e. CPU bound and IO bound.

These two tasks are very important in deciding what is the ideal thread pool size.

Newsletter Updates

Enter your name and email address below to subscribe to our newsletter

Leave a Reply