Thread Pool in Java

Why Use Thread Pools in Java?

In traditional Java multithreading, developers often create new threads using the Thread class. While this works for simple cases, it becomes inefficient when handling large-scale tasks like server request handling or background job execution. Creating a new thread every time a task needs to run can lead to high memory usage, CPU overhead, and performance degradation—especially when thousands of threads are created and destroyed frequently.

To solve this, Java provides the Thread Pool in Java—a pool of pre-created, reusable threads that efficiently handle multiple tasks. This is a core concept in Java concurrency, and it's part of the java.util.concurrent package, typically implemented using ExecutorService.

Key Reasons to Use Thread Pools in Java

• Performance Efficiency:
  Creating a thread is a time-consuming process. A Thread Pool in Java reuses existing threads, reducing the time needed to start tasks, especially in high-throughput systems.
   
• Resource Optimization:
  Thread pools limit the number of threads running at any time, preventing resource exhaustion. This is crucial in scenarios like web servers, where each user request may use a separate thread.
   
• Reusability of Threads:
  Thread reuse in Java avoids the overhead of constantly creating and destroying threads. Threads are recycled after completing their tasks, enhancing overall application performance.
   
• Improved Scalability and System Stability:
  By managing the number of active threads, thread pools make applications more stable and scalable, especially under heavy load. This improves the reliability of systems built on Java concurrency.

Executor Thread Pools Methods

The `java.util.concurrent` package provides several methods to create & manage Thread Pools through the `Executors` class. Here are some commonly used methods:

1. `newFixedThreadPool(int nThreads)`: Creates a Thread Pool with a fixed number of worker threads. If all threads are busy & a new task is submitted, it waits in a queue until a thread becomes available.
 

2. `newCachedThreadPool()`: Creates a Thread Pool that creates new threads as needed, but reuses previously constructed threads when they are available. Idle threads are kept for 60 seconds before being terminated & removed from the pool.
 

3. `newSingleThreadExecutor()`: Creates a Thread Pool with only one worker thread. This ensures that tasks are executed sequentially in the order they are submitted.
 

4. `newScheduledThreadPool(int corePoolSize)`: Creates a Thread Pool that can schedule tasks to run after a given delay or periodically.

Let’s look at an example demonstrating the usage of `newCachedThreadPool()`:

• Java

Java

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class CachedThreadPoolExample {
    public static void main(String[] args) {
        ExecutorService executorService = Executors.newCachedThreadPool();
        
        for (int i = 0; i < 10; i++) {
            Runnable task = new Task(i);
            executorService.execute(task);
        }
        
        executorService.shutdown();
    }
}

class Task implements Runnable {
    private int taskId;
    
    public Task(int taskId) {
        this.taskId = taskId;
    }
    
    @Override
    public void run() {
        System.out.println("Task " + taskId + " is running on thread " + Thread.currentThread().getName());
    }
}

You can also try this code with Online Java Compiler

Run Code

Output

Task 1 is running on thread pool-1-thread-2
Task 3 is running on thread pool-1-thread-4
Task 7 is running on thread pool-1-thread-8
Task 9 is running on thread pool-1-thread-10
Task 6 is running on thread pool-1-thread-7
Task 5 is running on thread pool-1-thread-6
Task 4 is running on thread pool-1-thread-5
Task 8 is running on thread pool-1-thread-9
Task 2 is running on thread pool-1-thread-3
Task 0 is running on thread pool-1-thread-1

In this example, we create a `CachedThreadPool` using `Executors.newCachedThreadPool()`. The Thread Pool creates new threads as needed to handle the submitted tasks. If an idle thread is available when a task is submitted, it is reused. This flexibility allows the Thread Pool to adapt to the workload dynamically.

ThreadPoolExecutor vs ExecutorService in Java

The Java Executor framework simplifies multithreading by decoupling task submission from thread management. Two common components of this framework are ExecutorService and ThreadPoolExecutor, often compared in the context of Java thread pool implementations.

What is ExecutorService?

ExecutorService is a high-level interface in the java.util.concurrent package. It represents an abstraction for executing tasks asynchronously using a thread pool.

ExecutorService example:

ExecutorService executor = Executors.newFixedThreadPool(3);
executor.submit(() -> System.out.println("Task executed"));
executor.shutdown();

You can also try this code with Online Java Compiler

Run Code

What is ThreadPoolExecutor?

ThreadPoolExecutor is a low-level concrete implementation of ExecutorService, offering advanced control over the thread pool's behavior and configuration.

ThreadPoolExecutor example:

ExecutorService executor = new ThreadPoolExecutor(
    2, 4, 60, TimeUnit.SECONDS,
    new LinkedBlockingQueue<>()
);
executor.execute(() -> System.out.println("Custom thread pool task"));
executor.shutdown();

You can also try this code with Online Java Compiler

Run Code

Key Differences Between ThreadPoolExecutor and ExecutorService

When to Use Which?

Use ExecutorService when:

• You need to quickly manage asynchronous tasks without worrying about internal thread management.
   
• You're building a simple application, like processing small background jobs or short-lived tasks.
   
• Example: Submitting image resizing tasks or logging operations without blocking the main thread.

Use ThreadPoolExecutor when:

• You need fine-grained control over thread count, queue capacity, or custom rejection policies.
   
• You're designing a performance-critical system, like a high-traffic web server or messaging queue processor.
   
• Example: Handling thousands of concurrent API requests where tuning thread usage directly impacts performance.

Thread Pool Example

Let's look at a more practical example of using a Thread Pool in Java. Suppose we have a list of files that need to be processed. We can use a Thread Pool to process these files concurrently, improving the overall performance.

• Java

Java

import java.io.File;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class FileProcessingExample {
    public static void main(String[] args) {
        // Create a list of files to process
        List<File> files = new ArrayList<>();
        files.add(new File("file1.txt"));
        files.add(new File("file2.txt"));
        files.add(new File("file3.txt"));
        
        // Create a fixed-size Thread Pool with 3 worker threads
        ExecutorService executorService = Executors.newFixedThreadPool(3);
        
        // Submit file processing tasks to the Thread Pool
        for (File file : files) {
            Runnable task = new FileProcessor(file);
            executorService.execute(task);
        }
        
        // Shutdown the Thread Pool after all tasks are submitted
        executorService.shutdown();
    }
}

class FileProcessor implements Runnable {
    private File file;
    
    public FileProcessor(File file) {
        this.file = file;
    }
    
    @Override
    public void run() {
        // Process the file
        System.out.println("Processing file: " + file.getName());
        
        // Simulate file processing time
        try {
            Thread.sleep(2000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        
        System.out.println("File processed: " + file.getName());
    }
}

You can also try this code with Online Java Compiler

Run Code

Output

Processing file: file2.txt
Processing file: file1.txt
Processing file: file3.txt
File processed: file1.txt
File processed: file2.txt
File processed: file3.txt

In this example, we have a list of files that need to be processed. We create a fixed-size Thread Pool with 3 worker threads using `Executors.newFixedThreadPool(3)`. We then iterate over the list of files & submit a `FileProcessor` task for each file to the Thread Pool using `executorService.execute(task)`.

The `FileProcessor` class implements the `Runnable` interface & represents the task of processing a file. Inside the `run()` method, we simulate file processing by printing a message & adding a delay of 2 seconds using `Thread.sleep(2000)`.

The Thread Pool manages the execution of these tasks concurrently, utilizing the available worker threads. As soon as a worker thread completes processing a file, it becomes available to handle the next file in the queue.

Risks in using Thread Pools

While Thread Pools has many advantages, just like any other tool, they have some risks also, like:

1. Thread Leakage: If tasks submitted to the Thread Pool have uncaught exceptions or don't complete properly, the threads may become stuck, leading to thread leakage. This can eventually cause the Thread Pool to run out of available threads, resulting in a program that hangs or crashes.
 

2. Resource Thrashing: If the Thread Pool is not sized appropriately or if tasks are not efficiently designed, it can lead to resource thrashing. This happens when threads compete for limited resources, such as CPU or I/O, leading to excessive context switching & reduced overall performance.
 

3. Deadlocks: Improper use of thread synchronization or resource locking within tasks can lead to deadlocks. If two or more tasks are waiting for each other to release resources, it can result in a deadlock, where the tasks are stuck indefinitely.
 

4. Starvation: If tasks with lower priority or longer execution times are continuously added to the Thread Pool, they may starve other tasks of CPU time. This can happen if the Thread Pool is not configured to handle tasks fairly, leading to some tasks being delayed or never executed.

To mitigate these risks, it's important to follow best practices when using Thread Pools:

- Properly handle exceptions within tasks to prevent thread leakage.

- Size the Thread Pool appropriately based on the system resources & expected workload.

- Design tasks efficiently to minimize resource contention & lengthy operations.

- Use thread synchronization mechanisms, such as locks & semaphores, carefully to avoid deadlocks.

- Consider using a fair task execution policy or a priority-based queue to prevent starvation.

Let’s look at an example of how to handle exceptions within a task:

class ExceptionHandlingTask implements Runnable {
    @Override
    public void run() {
        try {
            // Task logic that may throw an exception
            throw new IllegalArgumentException("Example exception");
        } catch (Exception e) {
            // Handle the exception gracefully
            System.err.println("Exception occurred: " + e.getMessage());
        }
    }
}

In this example, the task logic is enclosed within a try-catch block. If an exception occurs, it is caught & handled gracefully by printing an error message. This prevents the exception from propagating & causing thread leakage.

Best Practices and Important Considerations for Using Thread Pools in Java

When working with Thread Pools in Java, there are several important points to keep in mind:

1. Thread Pool Sizing: Choosing the appropriate size for a Thread Pool is crucial. If the pool is too small, it may lead to underutilization of system resources & increased task queuing. If the pool is too large, it may lead to excessive context switching & reduced performance. Consider factors such as the number of available cores, expected task duration, & system load when determining the optimal pool size.
 

2. Task Submission: Tasks can be submitted to a Thread Pool using the `execute()` method for simple Runnable tasks or the `submit()` method for Callable tasks that return a result. The `submit()` method returns a `Future` object, which can be used to retrieve the result or check the status of the task.
 

3. Task Completion: To ensure proper completion of tasks & graceful shutdown of the Thread Pool, it's important to call the `shutdown()` method after all tasks have been submitted. This prevents new tasks from being accepted & allows previously submitted tasks to complete. The `awaitTermination()` method can be used to wait for all tasks to finish before proceeding further.
 

4. Monitoring: Monitoring the Thread Pool is essential to ensure its health & performance. The `ThreadPoolExecutor` class provides methods to monitor the pool's status, such as `getActiveCount()` to get the number of currently executing tasks, `getCompletedTaskCount()` to get the number of completed tasks, & `getQueue()` to access the task queue.
 

5. Exception Handling: Proper exception handling within tasks is crucial to prevent thread leakage & ensure the stability of the Thread Pool. Exceptions should be caught & handled appropriately within the task's `run()` method. Uncaught exceptions can cause threads to abruptly terminate, leading to resource leaks.

Here's an example showing some of these important points in a code, to clear any doubt of yours:

• Java

Java

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class ThreadPoolMonitoringExample {
    public static void main(String[] args) throws InterruptedException {
        ThreadPoolExecutor executorService = (ThreadPoolExecutor) Executors.newFixedThreadPool(3);
        
        // Submit tasks to the Thread Pool
        for (int i = 0; i < 5; i++) {
            executorService.submit(new Task(i));
        }
        
        // Monitor the Thread Pool
        while (true) {
            System.out.println("Active Threads: " + executorService.getActiveCount());
            System.out.println("Completed Tasks: " + executorService.getCompletedTaskCount());
            System.out.println("Task Queue Size: " + executorService.getQueue().size());
            
            if (executorService.getActiveCount() == 0 && executorService.getQueue().isEmpty()) {
                break;
            }
            
            TimeUnit.SECONDS.sleep(1);
        }
        
        // Shutdown the Thread Pool
        executorService.shutdown();
        executorService.awaitTermination(1, TimeUnit.MINUTES);
    }
}

class Task implements Runnable {
    private int taskId;
    
    public Task(int taskId) {
        this.taskId = taskId;
    }
    
    @Override
    public void run() {
        System.out.println("Task " + taskId + " is running.");
        
        try {
            TimeUnit.SECONDS.sleep(2);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        
        System.out.println("Task " + taskId + " completed.");
    }
}

You can also try this code with Online Java Compiler

Run Code

Output

Task 0 is running.
Task 1 is running.
Task 2 is running.
Active Threads: 3
Completed Tasks: 0
Task Queue Size: 2
Task 0 completed.
Task 3 is running.
Task 1 completed.
Task 4 is running.
Active Threads: 3
Completed Tasks: 2
Task Queue Size: 0
Task 2 completed.
Active Threads: 2
Completed Tasks: 3
Task Queue Size: 0
Task 3 completed.
Active Threads: 1
Completed Tasks: 4
Task Queue Size: 0
Task 4 completed.
Active Threads: 0
Completed Tasks: 5
Task Queue Size: 0

In this example, we create a fixed-size Thread Pool with 3 worker threads & submit 5 tasks to it. We then monitor the Thread Pool using a loop that prints the number of active threads, completed tasks, & the size of the task queue. The loop continues until all tasks are completed & the task queue is empty.

After all tasks are completed, we shut down the Thread Pool using the `shutdown()` method & wait for up to 1 minute for all tasks to finish using the `awaitTermination()` method.

Tuning Thread Pools

Tuning a Thread Pool involves adjusting its configuration to optimize performance based on the specific requirements of the application. Here are some key aspects to consider when tuning a Thread Pool:

1. Pool Size: The size of the Thread Pool determines the maximum number of threads that can be active simultaneously. Setting the pool size too low may lead to underutilization of system resources, while setting it too high may result in excessive context switching & overhead. A good starting point is to set the pool size equal to the number of available CPU cores. However, this may need to be adjusted based on the specific workload & characteristics of the tasks.

2. Queue Size: The queue size determines the maximum number of tasks that can be queued waiting for an available thread. If the queue size is too small, tasks may be rejected if the queue is full. If the queue size is too large, it may consume excessive memory & lead to increased latency. The appropriate queue size depends on the expected task arrival rate & the processing time of each task.
 

3. Rejection Policy: When the Thread Pool & queue are full, the rejection policy determines how new tasks are handled. Common rejection policies include:

   - `AbortPolicy`: Throws a `RejectedExecutionException` if a task is rejected.

   - `CallerRunsPolicy`: The calling thread itself executes the rejected task.

   - `DiscardPolicy`: Silently discards the rejected task without executing it.

   - `DiscardOldestPolicy`: Discards the oldest unhandled task in the queue & tries to resubmit the new task.
 

4. Keep-Alive Time: For Thread Pools that dynamically adjust their size, such as the `CachedThreadPool`, the keep-alive time specifies how long idle threads should be kept in the pool before being terminated. Setting the keep-alive time too short may result in frequent thread creation & destruction, while setting it too long may lead to resource wastage.

Let’s look at an example of creating a custom Thread Pool with customised parameters:

• Java

Java

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class TunedThreadPoolExample {
    public static void main(String[] args) {
        int corePoolSize = 5;
        int maximumPoolSize = 10;
        long keepAliveTime = 60;
        TimeUnit unit = TimeUnit.SECONDS;
        BlockingQueue<Runnable> workQueue = new LinkedBlockingQueue<>(100);
        
        ThreadPoolExecutor executor = new ThreadPoolExecutor(
                corePoolSize,
                maximumPoolSize,
                keepAliveTime,
                unit,
                workQueue,
                new ThreadPoolExecutor.CallerRunsPolicy()
        );
        
        // Submit tasks to the tuned Thread Pool
        for (int i = 0; i < 50; i++) {
            executor.execute(new Task(i));
        }
        
        executor.shutdown();
    }
}

class Task implements Runnable {
    private int taskId;
    
    public Task(int taskId) {
        this.taskId = taskId;
    }
    
    @Override
    public void run() {
        System.out.println("Task " + taskId + " is running.");
        
        try {
            TimeUnit.SECONDS.sleep(1);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        
        System.out.println("Task " + taskId + " completed.");
    }
}

You can also try this code with Online Java Compiler

Run Code

Output

Task 2 is running.
Task 4 is running.
Task 0 is running.
Task 1 is running.
Task 3 is running.
Task 4 completed.
Task 1 completed.
Task 5 is running.
Task 0 completed.
Task 7 is running.
Task 3 completed.
Task 8 is running.
Task 6 is running.
Task 2 completed.
Task 9 is running.
Task 5 completed.
Task 7 completed.
Task 10 is running.
Task 11 is running.
Task 8 completed.
Task 12 is running.
Task 6 completed.
Task 13 is running.
Task 9 completed.
Task 14 is running.
Task 10 completed.
Task 11 completed.
Task 15 is running.
Task 12 completed.
Task 16 is running.
Task 17 is running.
Task 13 completed.
Task 18 is running.
Task 14 completed.
Task 19 is running.
Task 15 completed.
Task 20 is running.
Task 16 completed.
Task 17 completed.
Task 21 is running.
Task 22 is running.
Task 18 completed.
Task 23 is running.
Task 19 completed.
Task 24 is running.
Task 20 completed.
Task 25 is running.
Task 22 completed.
Task 21 completed.
Task 27 is running.
Task 23 completed.
Task 28 is running.
Task 26 is running.
Task 24 completed.
Task 29 is running.
Task 25 completed.
Task 30 is running.
Task 27 completed.
Task 31 is running.
Task 28 completed.
Task 32 is running.
Task 26 completed.
Task 33 is running.
Task 29 completed.
Task 34 is running.
Task 30 completed.
Task 31 completed.
Task 35 is running.
Task 32 completed.
Task 36 is running.
Task 33 completed.
Task 37 is running.
Task 34 completed.
Task 38 is running.
Task 39 is running.
Task 35 completed.
Task 36 completed.
Task 40 is running.
Task 41 is running.
Task 37 completed.
Task 42 is running.
Task 38 completed.
Task 39 completed.
Task 43 is running.
Task 44 is running.
Task 40 completed.
Task 41 completed.
Task 45 is running.
Task 46 is running.
Task 42 completed.
Task 47 is running.
Task 43 completed.
Task 48 is running.
Task 44 completed.
Task 49 is running.
Task 45 completed.
Task 46 completed.
Task 47 completed.
Task 49 completed.
Task 48 completed.

In this example, we create a custom `ThreadPoolExecutor` with tuned parameters. We set the core pool size to 5, the maximum pool size to 10, the keep-alive time to 60 seconds, & the queue size to 100. We also specify the `CallerRunsPolicy` as the rejection policy.

We then submit 50 tasks to the tuned Thread Pool. The Thread Pool will dynamically adjust its size based on the workload, up to the maximum pool size. If the queue is full & all threads are busy, the `CallerRunsPolicy` will cause the calling thread to execute the rejected task directly.

Tuning the Thread Pool requires careful consideration of the application's requirements, workload characteristics, & system resources. It may involve iterative testing & monitoring to find the optimal configuration for the specific use case.

Advantages of Thread Pools

1. Improved Performance

Thread pools improve performance by reusing existing threads instead of creating a new thread for each task. This reduces the overhead of thread creation and destruction. For example, a server handling multiple client requests can respond faster using a thread pool.
 

2. Better Resource Management

Thread pools limit the number of active threads, preventing system overload. This ensures better control over CPU and memory usage. It's like using only a few cashiers at a store even if many customers arrive, avoiding chaos and system crashes.
 

3. Scalability

Thread pools make applications scalable by efficiently handling increasing workloads. As more tasks come in, the pool adjusts within limits, maintaining stable performance. Web servers benefit from this by managing thousands of requests without spawning thousands of threads.
 

4. Reduced Latency

Reusing threads leads to faster task execution, minimizing the delay between request and response. Since threads are already available, tasks don't wait long to start, which is helpful in real-time systems like online trading platforms.
 

5. Centralized Thread Control

Thread pools provide a central mechanism to manage thread behavior, such as maximum thread count or task queues. This simplifies control and monitoring, making it easier to adjust performance settings for applications like messaging systems or batch processors.
 

6. Supports Future and Callable

Thread pools work well with Callable and Future, allowing tasks to return results or throw exceptions. For instance, background jobs that return a value (like a file download status) can be tracked easily using these interfaces.

Disadvantages of Thread Pools

1. Complexity in Configuration

Choosing the right settings (like core size, max size, and queue type) is tricky. Incorrect configuration can lead to poor performance. For example, setting too many threads can overwhelm the system; too few can slow down processing.
 

2. Potential for Resource Starvation

If the thread pool size is too small and tasks are waiting in a long queue, some may never get executed. This is called resource starvation, and it can freeze parts of your application, especially when tasks depend on each other.
 

3. Debugging Difficulty

Thread pools can make debugging harder. When many threads are active in the background, it’s challenging to trace where a task failed or got stuck. Logs and stack traces may not clearly show the root cause, especially in concurrent environments.
 

4. Risk of Thread Leaks

If tasks never finish or are blocked indefinitely, the threads handling them remain occupied—this is known as a thread leak. Over time, available threads decrease, leading to stalled applications, similar to memory leaks but with threads.
 

5. Improper Shutdown Issues

Forgetting to properly shut down a thread pool using shutdown() or shutdownNow() can cause applications to hang. Background threads may keep running, preventing the program from exiting. This often happens in desktop or testing applications.

Frequently Asked Questions

What happens if all threads in the Thread Pool are busy & a new task is submitted?

If all threads are busy, the new task is added to the work queue. If the queue is full, the rejection policy determines how the task is handled.

Can a Thread Pool be resized dynamically during runtime?

Yes, the ThreadPoolExecutor class allows dynamic resizing of the pool using methods like setCorePoolSize() & setMaximumPoolSize().

How can I gracefully terminate a Thread Pool?

To gracefully terminate a Thread Pool, call the shutdown() method, which allows previously submitted tasks to be completed. Use awaitTermination() to wait for all tasks to finish.

Conclusion

In this article, we discussed Java Thread Pools, a powerful mechanism for managing & optimizing concurrent task execution. We learned about the benefits of using Thread Pools, such as improved performance, resource management, & simplified programming. We explained different methods provided by the Executors class to create Thread Pools & examined a practical example of processing files concurrently using a Thread Pool.

You can also practice coding questions commonly asked in interviews on Coding Ninjas Code360. 

Also, check out some of the Guided Paths on topics such as Data Structure and Algorithms, Competitive Programming, Operating Systems, Computer Networks, DBMS, System Design, etc., as well as some Contests, Test Series, and Interview Experiences curated by top Industry Experts.