Executors & Thread Pools — Where 90% of Java Concurrency Bugs Live¶

Creating threads by hand (new Thread(runnable).start()) is almost always wrong in production code. Threads are expensive, unbounded thread creation kills the JVM, and lifecycle management (waiting, cancelling, timing out) is a swamp. The Executor framework exists so you never write that code by hand.

This file covers the framework, the pool-sizing math, the dangerous factory methods, and the correct way to compose async work with CompletableFuture.

1. The Framework in One Diagram¶

  Executor           execute(Runnable)                     — fire and forget
     ↑
  ExecutorService    submit(Callable) → Future<T>          — add cancellation, shutdown, results
     ↑
  ScheduledExecutorService  schedule / scheduleAtFixedRate — add timed execution

  ThreadPoolExecutor       — the concrete class you'll actually configure
  ForkJoinPool             — work-stealing pool for divide-and-conquer

Everything the Executors factory returns is one of the above with different constructor arguments. Understanding ThreadPoolExecutor’s parameters lets you understand every factory method — and lets you build the right pool yourself when a factory is wrong for your workload (usually).

2. `ThreadPoolExecutor` — The One Constructor to Know¶

new ThreadPoolExecutor(
    int corePoolSize,               // threads always kept alive
    int maximumPoolSize,            // ceiling under load
    long keepAliveTime,             // idle time before non-core threads exit
    TimeUnit unit,
    BlockingQueue<Runnable> workQueue,   // where waiting tasks sit
    ThreadFactory threadFactory,         // names threads, sets daemon, uncaught handler
    RejectedExecutionHandler handler     // what happens when overloaded
);

Lifecycle of a submitted task:

If fewer than corePoolSize threads exist, create a new one and hand it the task.
Otherwise, try to enqueue into workQueue.
If the queue is full and thread count < maximumPoolSize, create a new thread.
If the queue is full and thread count == maximumPoolSize, invoke the RejectedExecutionHandler.

The order matters. The pool does not grow past corePoolSize until the queue is full. If you use an unbounded queue (LinkedBlockingQueue with no capacity), step 3 never fires — maximumPoolSize and keepAliveTime are dead code and you have a fixed pool with an infinite backlog.

3. The `Executors` Factory Methods — What They Actually Do¶

Factory	Under the hood	Danger
`newFixedThreadPool(n)`	`ThreadPoolExecutor(n, n, 0, MS, new LinkedBlockingQueue<>())`	Unbounded queue. OOM on backlog.
`newCachedThreadPool()`	`ThreadPoolExecutor(0, MAX_VALUE, 60, SEC, new SynchronousQueue<>())`	Unbounded threads. OOM if tasks arrive faster than they complete.
`newSingleThreadExecutor()`	Same as fixed with n=1	Same unbounded-queue trap.
`newScheduledThreadPool(n)`	`ScheduledThreadPoolExecutor`	Silent task failure — see below.
`newWorkStealingPool()`	`ForkJoinPool` with parallelism = `availableProcessors()`	Fine, but not what you usually want for I/O.

⚠️ Rule: in production code, prefer building ThreadPoolExecutor yourself with a bounded queue and an explicit rejection policy. The factory methods are demo-quality. The Google Java Style Guide, Netflix, and Doug Lea himself have said as much.

4. Sizing a Thread Pool¶

Brian Goetz’s formula, from Java Concurrency in Practice:

threads  =  cores  ×  target CPU utilization  ×  (1 + wait time / compute time)

CPU-bound (compression, hashing, matrix math):

threads ≈ cores + 1. The +1 is a compromise for occasional page faults.
Going higher just increases context-switch overhead. You’ve saturated the CPUs.

I/O-bound (HTTP calls, JDBC, file writes):

Ratio matters. If a task spends 10ms computing and 90ms waiting on I/O, you can profitably run ~10× cores threads.
Real answer: measure. Load-test at increasing pool sizes and watch throughput plateau, then decline. That plateau is your number.
Or: use virtual threads and stop sizing pools for I/O. See file 05.

Mixed workload: split into two pools — one for CPU work, one for I/O work — so a burst of I/O doesn’t starve CPU-bound tasks and vice versa. This is called “bulkheading” and it prevents one slow downstream from bringing down your entire service.

5. Rejection Policies¶

When the queue is full and threads are at max, the pool must do something with the incoming task. ThreadPoolExecutor has four built-in policies:

Policy	Behavior	Use when
`AbortPolicy` (default)	Throws `RejectedExecutionException`	You want the caller to know. Sane default.
`CallerRunsPolicy`	Runs the task on the submitting thread	You want backpressure — slows down the producer.
`DiscardPolicy`	Silently drops the task	Almost never. Only for telemetry-style workloads where drops are acceptable.
`DiscardOldestPolicy`	Drops the oldest queued task, retries submit	Rare. Real-time systems maybe.

CallerRunsPolicy is the underappreciated one. Under overload, the submitter’s own thread executes the task, which naturally throttles it. For an HTTP request handler, that means the request stalls (bad) but the pool doesn’t grow unbounded (worse).

6. `Future` vs `CompletableFuture`¶

`Future<T>` — the original¶

Future<Integer> f = executor.submit(() -> compute());
Integer result = f.get(5, TimeUnit.SECONDS);   // blocking, with timeout

Problems: get() blocks, you cannot chain, you cannot combine futures without more blocking. Fine for fire-and-collect, useless for a pipeline.

`CompletableFuture<T>` — the composable one¶

CompletableFuture<User>  userFuture   = CompletableFuture.supplyAsync(() -> fetchUser(id), executor);
CompletableFuture<Order> orderFuture  = userFuture.thenApplyAsync(user -> fetchOrder(user), executor);
CompletableFuture<Void>  done         = orderFuture.thenAcceptAsync(this::render, executor);

done.exceptionally(ex -> {
    log.error("pipeline failed", ex);
    return null;
});

Rules of CompletableFuture that trip people up:

Always pass an explicit executor to the Async variants. Without it, work runs on the common ForkJoinPool, which is shared with parallel streams and sized to cores - 1. One slow HTTP call blocks parallel streams elsewhere.
thenApply vs thenApplyAsync: thenApply runs the continuation on whatever thread completed the previous stage — which might be your I/O thread. thenApplyAsync moves it back to an executor. Use Async when the continuation does non-trivial work.
Exceptions don’t propagate like regular Java. They surface via exceptionally, handle, or whenComplete. Forgetting one of these means silent failures.
get() still blocks. If you’re chaining futures, don’t call get() — chain another stage.

Combining futures:

CompletableFuture<A> fa = ...;
CompletableFuture<B> fb = ...;

// Wait for both, combine results:
CompletableFuture<C> fc = fa.thenCombine(fb, (a, b) -> combine(a, b));

// Wait for either (whichever is faster):
CompletableFuture<Object> either = CompletableFuture.anyOf(fa, fb);

// Wait for all N:
CompletableFuture<Void> all = CompletableFuture.allOf(f1, f2, f3);
List<Result> results = all.thenApply(v ->
    Stream.of(f1, f2, f3).map(CompletableFuture::join).collect(toList())
).get();

7. `ScheduledExecutorService` — Timed Tasks¶

ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(2);

// Once, after delay:
scheduler.schedule(task, 30, TimeUnit.SECONDS);

// Fixed rate: next run starts 60s after this run STARTS (may overlap under load)
scheduler.scheduleAtFixedRate(task, 0, 60, TimeUnit.SECONDS);

// Fixed delay: next run starts 60s after this run FINISHES
scheduler.scheduleWithFixedDelay(task, 0, 60, TimeUnit.SECONDS);

⚠️ The silent-failure trap: if a scheduled task throws an uncaught exception, subsequent executions never run. The pool doesn’t restart it, and it doesn’t log. Always wrap scheduled tasks in a try/catch and log failures explicitly. This has burned real engineers on real production incidents; do not skip it.

scheduler.scheduleAtFixedRate(() -> {
    try {
        doWork();
    } catch (Throwable t) {
        log.error("scheduled task failed", t);
    }
}, 0, 60, TimeUnit.SECONDS);

For “fixed rate vs fixed delay”: if you don’t want overlap, always use scheduleWithFixedDelay. It’s the safer default.

8. Shutdown, Cleanly¶

executor.shutdown();                       // stop accepting new tasks
if (!executor.awaitTermination(30, TimeUnit.SECONDS)) {
    executor.shutdownNow();                // interrupt running tasks
    if (!executor.awaitTermination(10, TimeUnit.SECONDS)) {
        log.error("executor did not terminate");
    }
}

Register this in a JVM shutdown hook or a Spring @PreDestroy. Forgetting to shut down a non-daemon executor keeps the JVM alive after main returns — the classic “why isn’t my process exiting?” bug.

9. `ThreadFactory` — Name Your Threads¶

ThreadFactory factory = Thread.ofPlatform()
    .name("http-worker-", 0)               // http-worker-0, http-worker-1, ...
    .daemon(false)
    .uncaughtExceptionHandler((t, e) -> log.error("uncaught in " + t.getName(), e))
    .factory();

ExecutorService pool = new ThreadPoolExecutor(
    8, 32, 60L, SECONDS,
    new ArrayBlockingQueue<>(1000),
    factory,
    new ThreadPoolExecutor.CallerRunsPolicy()
);

When you take a thread dump at 3 AM, thread names are the difference between finding your bug in 30 seconds and 30 minutes. Never accept the default pool-1-thread-3 names in production.

10. Decision Table¶

Workload	Pool
CPU-bound compute	`ThreadPoolExecutor`, size = cores + 1, bounded queue
I/O-bound (pre-Loom)	`ThreadPoolExecutor`, size = measured, bounded queue
I/O-bound (Java 21+, safe libraries)	Virtual thread `Executors.newVirtualThreadPerTaskExecutor()`
Divide-and-conquer (`Stream.parallel()`, recursive tasks)	`ForkJoinPool`
Scheduled/periodic	`ScheduledThreadPoolExecutor`
Async pipelines	`CompletableFuture` + one of the above

⚠️ What Most People Get Wrong¶

They call Executors.newFixedThreadPool(200) in a REST handler and call it done. Then a downstream service slows down, tasks pile up in the unbounded queue, heap fills with Runnables, and the service OOMs while the CPU sits at 5%. The fix is trivial once you know: bounded queue, CallerRunsPolicy, small named pool. But by default nobody thinks about the queue — they think about the threads.

Also: they chain CompletableFuture on the common ForkJoinPool without knowing it. Every parallel stream in the JVM competes for the same pool. Diagnose it once, remember it forever: always pass an explicit executor to Async methods.

Return to README.md · Next: 04_concurrent_collections.md

Executors & Thread Pools — Where 90% of Java Concurrency Bugs Live¶

1. The Framework in One Diagram¶

2. ThreadPoolExecutor — The One Constructor to Know¶

3. The Executors Factory Methods — What They Actually Do¶