How to Build Low Latency Java Applications: Good Practices for 2026

A pricing service answers in 300 microseconds at the median, yet every few seconds one quote takes 40 milliseconds and the client has already traded elsewhere. Low latency Java applications are judged by that slow tail.

Building one means measuring the tail honestly, removing every source of waiting and jitter from the hot path, and paying for the exotic techniques only where the latency budget demands them.

Introduction to latency

Latency is the time between a request, or an event such as a price change, and the moment the system has finished responding to it. Throughput is how many of those the system completes per second. A system can do well on one and badly on the other: a batch pipeline that moves a million records a second may still take minutes to deliver any single one.

The two are linked through queueing. As a server gets busier, requests wait longer for a free thread, a lock or a network buffer, and latency climbs steeply well before throughput reaches its limit. Most latency problems turn out to be waiting rather than slow code; our list of common Java performance problems covers the usual causes.

Why the tail matters more than the average

An average hides the requests users complain about, so latency is reported in percentiles. A p99 of 5 ms means 99 requests in 100 finish within 5 ms and one in 100 takes longer. At 10,000 requests a second, one in 100 is a hundred slow responses every second.

The tail also compounds. In The Tail at Scale, Google's Jeffrey Dean and Luiz André Barroso take a server that typically answers in 10 ms but has a p99 of one second. A request that has to collect answers from 100 such servers in parallel is slow whenever any one of them is, so 63% of user requests take more than a second.

The same arithmetic applies to a page that makes 30 calls to one service: it meets that service's p99 on about 26% of page loads.

Latency numbers every programmer should know

The list that circulates as "latency numbers every programmer should know" comes from Peter Norvig's essay Teach Yourself Programming in Ten Years and from Jeff Dean's talks at Google. The figures date from early-2000s hardware. SSDs have since closed much of the gap to disk and networks have become faster, but the orders of magnitude still hold, and a latency budget is built from them.

Read the chart with a budget in mind. A main memory reference costs about 100 ns, so a hot path that misses the CPU cache 1,000 times has spent 100 µs before doing anything useful. A network round trip to a database, even inside one data center, costs more than the computation in most request handlers. A low-latency hot path stays at the top of this chart: in cache, in memory, on one machine.

Set a latency budget before you optimize anything

A latency target that says "fast" cannot be tested. A usable one names a percentile, a load and the point of measurement: p99.9 under 1 ms at 50,000 requests a second, measured at the client. Without the load figure, any system meets its target at idle. Without the measurement point, a team can meet it inside the JVM while requests wait in a queue in front of it.

The next step is to decide which latency the business pays for. For Mennica Skarbowa, a Polish precious-metals distributor, we built a real-time pricing engine fed by London Metal Exchange and distributor quotations. Physical gold is a low-margin, high-price business, and a price published late is sold at a loss, so the latency that cost money there was the time from a market move to a new price in every web store and physical shop. The budget covered the whole path: loading the quotations, recalculating margins by product and channel, and distributing the prices.

Once the target is set, split it across the stages of the path and keep part of it in reserve. GC cycles, safepoints, page faults and the operating system's scheduler add delays that no stage owns, and a budget that assigns all of its time to stages will be missed at the tail.

Splitting the budget also shows where the large wins are. A UK hotel metasearch engine we rebuilt was serving about 200 million queries a day, 99% of them reads, and every query went to a cluster of dozens of SQL Server machines. Adding a machine raised cost by about 2% and throughput by about 0.1%.

We moved availability search into an in-memory data grid that holds everything a query needs, so the request path no longer touches the database. Response time fell by 60%; the engine now serves 300 million queries a day with 95% answered in under a second, at 80% lower infrastructure cost. No JVM flag removes a network hop, which is why architecture comes before tuning. Our explainer on what Hazelcast is used for shows how such grids partition data, and the general method for finding a bottleneck is in our guide to improving Java performance.

Which techniques each latency budget calls for

The techniques in the rest of this guide have costs: cores that spin at 100%, code that avoids ordinary Java idioms, operational work on the hosts. The budget decides how many of them a system needs.

Most business systems sit in the first two rows, where measurement, data placement and a current collector do most of the work.

How to measure latency percentiles correctly

Most latency numbers from load tests are wrong in the same direction: they make the tail look better than it is. Two mistakes cause most of the error, averaging percentiles and coordinated omission.

Record whole distributions with HdrHistogram

A percentile cannot be averaged. The mean of ten hosts' p99 values is not the fleet's p99, and the mean of sixty one-second p99 values is not the p99 of the minute. Record the full distribution, merge distributions, and read percentiles from the merged result.

HdrHistogram, written by Gil Tene, records values across a wide range with a configurable number of significant digits and a fixed memory footprint, cheaply enough to sit on a hot path (HdrHistogram).

// nanoseconds, up to 10 seconds, 3 significant digits
Histogram latency = new Histogram(TimeUnit.SECONDS.toNanos(10), 3);

long start = System.nanoTime();
handle(request);
latency.recordValue(System.nanoTime() - start);

long p999 = latency.getValueAtPercentile(99.9);
latency.outputPercentileDistribution(System.out, 1000.0);   // report in microseconds

Histogram is not thread-safe. With many recording threads, a Recorder hands out interval histograms that a reporting thread merges and logs without stopping the writers.

Coordinated omission: why load tests under-report the tail

Gil Tene named the second mistake coordinated omission in his talk How NOT to Measure Latency. A closed-loop load generator sends a request, waits for the response, then sends the next. When the server stalls, the generator stalls with it. It records one slow response and never records the requests that real users would have sent during the stall, all of which would have waited.

The chart shows a model of a 60-second test at 1,000 requests a second with one 1-second stall in the middle. The generator that waits reports a p99 of 0.36 ms. Measured from the time each request should have been sent, the same run has a p99 of 525 ms, and the mean rises from 0.22 ms to 10.7 ms.

The correction is to measure from the intended send time. wrk2 sends at a constant rate set with -R and measures each response from the moment the request should have gone out, not from when it was actually sent (wrk2). HdrHistogram's recordValueWithExpectedInterval() back-fills the missing samples when the expected interval between requests is known.

Load generators and benchmarks that avoid it

For whole-system tests, use a generator with an open workload model, where arrivals do not depend on responses: wrk2 with -R, Gatling's open injection profiles such as constantUsersPerSec, or k6's constant-arrival-rate executor. Run it from a separate machine, long enough to include several GC cycles, at the load the target names.

wrk -t4 -c64 -d300s -R20000 --latency http://quote-host:8080/quote

For a single method, JMH in SampleTime mode reports percentiles of individual calls, and its -prof gc profiler reports bytes allocated per operation, which the next section depends on. JMH measures code in isolation and says nothing about queueing under load.

@BenchmarkMode(Mode.SampleTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Warmup(iterations = 5, time = 2)
@Measurement(iterations = 5, time = 5)
@Fork(value = 3, jvmArgs = {"-XX:+UseZGC", "-XX:+AlwaysPreTouch"})
@State(Scope.Thread)
public class QuoteBenchmark {
    private final PricingEngine engine = PricingEngine.withTestBook();
    private final QuoteRequest request = QuoteRequest.sample();

    @Benchmark
    public long quote() {
        return engine.quote(request);   // returned, so the JIT cannot remove the call
    }
}
// java -jar target/benchmarks.jar QuoteBenchmark -prof gc

Measure the platform too. jHiccup runs as a Java agent, wakes up at a fixed interval and records how late each wake-up was. Any delay it sees is a stall every thread in the process saw as well, whether it came from GC, a safepoint, the hypervisor or the kernel (jHiccup).

Garbage collection and allocation on the hot path

Garbage collection adds latency in two ways: through pauses, and through the CPU time and memory bandwidth that collection takes from the application. The choice of collector addresses the first. An allocation-free hot path reduces both.

Choosing a collector: generational ZGC, Shenandoah or G1

G1, the default collector, works to a pause-time goal (-XX:MaxGCPauseMillis, 200 ms unless set) and suits budgets of tens of milliseconds and more. Its pauses grow with the amount of live data it copies.

Generational ZGC does almost all of its work concurrently with the application. Its design goal is pauses that do not exceed one millisecond, on heaps from a few hundred megabytes to many terabytes (JEP 439). It arrived in Java 21 as an option, became ZGC's default mode in JDK 23 (JEP 474), and the non-generational mode was removed in JDK 24 (JEP 490). On Java 25, -XX:+UseZGC is all it takes.

Shenandoah, developed by Red Hat, also compacts the heap concurrently. Its generational mode was experimental in JDK 24 and became a product feature in Java 25 (JEP 521); JEP 535 makes it the default mode in JDK 28. Shenandoah ships in most OpenJDK builds, such as Temurin, Corretto and Red Hat's, but not in Oracle's JDK.

A concurrent collector still needs room. ZGC collects while the application allocates, and when allocation outruns it, application threads wait in allocation stalls, which JFR records as jdk.ZAllocationStall events and which show up in the tail like pauses. Give the heap headroom, fix its size and pre-touch it:

-XX:+UseZGC                      # Java 25: generational is the only ZGC mode
-Xms16g -Xmx16g                  # fixed heap, no resizing at run time
-XX:+AlwaysPreTouch              # fault in every heap page at startup, not on first use
-XX:+UseTransparentHugePages     # 2 MB pages for the heap (THP mode "madvise")
-XX:+UseCompactObjectHeaders     # Java 25 product feature, default from JDK 27
-Xlog:gc*,safepoint:file=gc.log:time,uptime,level,tags

Java 25 made compact object headers a product feature (JEP 519), and JDK 27 turns them on by default (JEP 534). Smaller headers mean smaller objects: in one setting, the JEP reports, the SPECjbb2015 benchmark used 22% less heap and 8% less CPU time. A smaller heap is collected less often.

Allocation-free hot paths

Allocation itself is cheap: a thread allocates from its own buffer (a TLAB) by bumping a pointer. The cost comes afterwards. Every allocated byte brings the next young collection closer, fills the cache with new lines and, under a concurrent collector, adds to the work running next to the application. At a million messages a second, 200 bytes per message is 200 MB of garbage every second.

The usual sources on a hot path are boxing (a Long key in a HashMap), iterators, lambdas that capture variables, varargs logging calls, string concatenation, and Optional or stream pipelines. Escape analysis removes some of these allocations, but only when everything involved is inlined, so verify with JMH's -prof gc, JFR's jdk.ObjectAllocationSample events or async-profiler's allocation mode.

// Allocates on most calls: a boxed key and an Optional
Optional<Order> find(long id) {
    return Optional.ofNullable(orders.get(id));    // HashMap<Long, Order>
}

// Allocation-free: a primitive-keyed map, null for "absent"
Order find(long id) {
    return orders.get(id);                         // Agrona Long2ObjectHashMap<Order>
}

Messages follow the same rule: decode into a mutable object reused for every message, or read fields straight from the buffer with a flyweight, as SBE does.

Object pooling: when it pays and when it hurts

Pooling small objects to avoid allocation usually loses on a current JVM. A pooled object survives collections and ends up in the old generation, and every reference it then holds to a young object adds work for the collector. Pools also bring back the bugs garbage collection removed: an object returned while still in use, or never returned.

Pooling pays for objects that are expensive to create or large, such as direct ByteBuffers (Netty pools them by default), connections and big arrays. The reused events in a ring buffer are a special case: a fixed set of objects, each with one owner at a time.

Off-heap memory with the FFM API

Data that is large, long-lived and simple in shape, such as an order book, a price cache or a lookup table with millions of entries, can live outside the heap, where the collector never scans it. The Foreign Function & Memory API, final since Java 22 (JEP 454), is the supported way to do this. It replaces the memory-access methods of sun.misc.Unsafe, deprecated for removal in JDK 23 (JEP 471).

// One million price levels outside the heap: 16 bytes each, 64-byte aligned
static final long LEVEL = 16;
final Arena arena = Arena.ofShared();                       // closed explicitly at shutdown
final MemorySegment levels = arena.allocate(LEVEL * 1_000_000, 64);

void set(int i, long price, long qty) {
    levels.set(ValueLayout.JAVA_LONG, i * LEVEL, price);
    levels.set(ValueLayout.JAVA_LONG, i * LEVEL + 8, qty);
}

long price(int i) {
    return levels.get(ValueLayout.JAVA_LONG, i * LEVEL);
}

Access is bounds-checked, and an arena's memory is freed when the arena closes. The price is a memory layout designed by hand and no objects on that path. FileChannel.map also returns a MemorySegment, which is how memory-mapped journals and queues are built.

JIT warmup: the Leyden AOT cache and warmup strategies

A freshly started JVM interprets bytecode, compiles hot methods with the C1 compiler while profiling them, and only then compiles them with C2 into the code a benchmark measures. Until then, requests run slower.

Later, when real traffic breaks an assumption C2 made, such as a branch never taken or a single receiver type at a call site, the JVM deoptimizes the method and the cycle starts again for that code. It shows up as a latency spike long after startup.

Project Leyden's AOT cache

Project Leyden moves part of this work into a training run whose results are stored in an AOT cache:

• JDK 24 caches loaded and linked classes (JEP 483). Spring PetClinic, which loads about 21,000 classes, started in 4.486 s on JDK 23 and in 2.604 s on JDK 24 with the cache, according to the JEP.
• Java 25 creates the cache in one step with -XX:AOTCacheOutput (JEP 514) and stores method profiles in it (JEP 515), so the JIT can compile hot methods right after startup. In the JEP's example, a short program ran in 73 ms instead of 90 ms.
• JDK 26 makes the cache's preloaded Java objects work with every collector, ZGC included (JEP 516).
• Ahead-of-time compiled code is targeted at JDK 28 (JEP 544).

# Training run: drive representative traffic, then stop the application
java -XX:AOTCacheOutput=quote.aot -jar quote-service.jar

# Production runs start from the cache
java -XX:AOTCache=quote.aot -XX:+UseZGC -jar quote-service.jar

The cache is valid only for the same JDK release and class path as the training run, so build it in the pipeline that builds the image.

Warmup strategies that hold up in production

An AOT cache shortens warmup. A hot path measured in microseconds still needs a warmed JIT before its first real request. Keep a new instance out of the load balancer until it has processed representative traffic: replayed production messages, or synthetic requests that take the same branches with the same types. Warming up with one repeated dummy request teaches the JIT the wrong profile and moves the deoptimization into production.

Two JFR events show whether warmup worked: jdk.Compilation for what was compiled and when, and jdk.Deoptimization for code that fell back to the interpreter. Forcing compilation with -Xcomp compiles every method without profile data and usually produces slower steady-state code. Azul's and BellSoft's JDK builds also offer CRaC, which restores a process from a checkpoint taken after warmup; it is not part of mainline OpenJDK.

Low-latency threading: single writer, Disruptor and pinned cores

The single-writer principle and the LMAX Disruptor

Martin Thompson's single-writer principle holds that the biggest limit on scalability is several writers contending for the same data. Locks make contending writers queue, often in the kernel. Compare-and-swap loops make them retry. Both move cache lines between cores. A design in which each piece of mutable state has exactly one writing thread needs neither.

The LMAX Disruptor, built for the LMAX exchange, applies the principle to passing messages between threads. Events live in a ring buffer preallocated at startup. A producer claims the next sequence number, fills the event in that slot and publishes the sequence. Each consumer tracks its own sequence and can be made to wait for others, so a pipeline of journaling, replication and business logic runs with no locks or queues between the stages.

In the Disruptor's technical paper, a three-stage pipeline measured a mean latency per hop of 52 ns, against 32,757 ns for ArrayBlockingQueue, and profiling put most of the difference down to locks and condition-variable signalling (LMAX Disruptor paper). The figures come from 2011 hardware and Java 6; the tail shows the gap more clearly than the mean:

A minimal pipeline with Disruptor 4, journaling and replication in parallel, then the business logic:

final class TickEvent {                  // one instance per slot, reused for every message
    long instrumentId;
    long priceMicros;
}

Disruptor<TickEvent> disruptor = new Disruptor<>(
        TickEvent::new, 65_536,          // ring size, a power of two
        DaemonThreadFactory.INSTANCE,
        ProducerType.SINGLE,             // one publishing thread
        new BusySpinWaitStrategy());
disruptor.handleEventsWith(journal, replicate).then(pricing);
RingBuffer<TickEvent> ring = disruptor.start();

// on the receiver thread
long seq = ring.next();
try {
    TickEvent e = ring.get(seq);
    e.instrumentId = instrumentId;
    e.priceMicros = price;
} finally {
    ring.publish(seq);
}

The producer writes into the claimed slot directly. The publishEvent variants that take lambda arguments are shorter to write, but their generic arguments box primitive values.

Busy-spin or blocking: what each wait strategy costs

A consumer with nothing to do either sleeps or spins. Waking a sleeping thread goes through the kernel scheduler and costs microseconds, more under load. A spinning thread sees new work within nanoseconds but keeps a core at 100% while it waits. The Disruptor user guide describes the options:

The user guide reserves busy spinning for deployments with fewer event-handler threads than physical cores. Spinning also fails badly in a container with a CPU limit: the spinning thread uses up the quota, the kernel throttles the container until the next scheduling period (100 ms by default), and a nanosecond wait becomes a pause of tens of milliseconds. In hand-written spin loops, call Thread.onSpinWait(), which lets the JIT emit the CPU's spin hint (PAUSE on x86).

Thread affinity and isolated cores

Spinning on a core the scheduler also gives to other processes defeats the purpose. On Linux, the kernel parameters isolcpus, nohz_full and rcu_nocbs take a set of cores out of general scheduling and away from most timer and RCU work. Latency-critical threads are then pinned to them, one thread per core, while GC, JIT and everything else run on the remaining cores.

# kernel command line: reserve cores 2 to 5 for pinned threads
isolcpus=2-5 nohz_full=2-5 rcu_nocbs=2-5

OpenHFT's Java-Thread-Affinity library binds a Java thread to one of the isolated cores:

try (AffinityLock lock = AffinityLock.acquireLock()) {
    runEventLoop();          // stays on one reserved core until the lock is released
}

The cost is capacity and operations: isolated cores do nothing else, and the setup lives in kernel parameters and deployment scripts that a container platform may not allow. It belongs to the bottom row of the budget table.

False sharing and @Contended

CPUs move memory in cache lines, usually 64 bytes. When two threads write to different fields that share a line, each write invalidates the other core's copy and the line moves back and forth between cores. This false sharing costs as much as real contention and is invisible in the source code.

final class Stats {
    @jdk.internal.vm.annotation.Contended volatile long received;    // written by the receiver
    @jdk.internal.vm.annotation.Contended volatile long processed;   // written by the processor
}
// compile: --add-exports java.base/jdk.internal.vm.annotation=ALL-UNNAMED
// run:     -XX:-RestrictContended   (otherwise the JVM ignores it outside the JDK)

@Contended is a JDK-internal annotation, so libraries pad by hand instead: the Disruptor and Agrona surround their sequence counters with unused long fields. Java profilers do not report false sharing; on Linux, perf c2c finds the cache lines involved.

Virtual threads: for I/O-bound latency, not hot loops

Virtual threads, final since Java 21, reduce latency in services that spend most of each request waiting on databases and other services, because requests stop queuing for a free platform thread. JDK 24 removed the main cause of carrier pinning, blocking inside synchronized (JEP 491). A CPU-bound hot loop gains nothing from them and gets a scheduler between the code and the core, so microsecond paths stay on dedicated platform threads. Our guide to Java virtual threads covers where they fit and what has gone wrong with them in production.

Data structures that stay in the CPU cache

On the latency scale, an L1 hit costs half a nanosecond and a main memory reference about 100. Choosing data structures for low latency is mostly about which of the two a lookup pays, and how often.

A HashMap<Long, Order> lookup follows a pointer to the bucket array, then to the entry node, the boxed key and the order, and each step can miss the cache. Primitive collections keep keys and values in flat arrays with open addressing, so a lookup touches one or two cache lines and allocates nothing. fastutil, Eclipse Collections and Agrona all provide them.

For data that is scanned rather than looked up, such as price levels or time series, parallel primitive arrays (a structure of arrays) keep the values a loop reads next to each other, where the CPU's prefetcher can stream them:

// Array of objects: each Level is a separate object somewhere on the heap
Level[] levels;

// Structure of arrays: contiguous primitives, read sequentially
long[] prices = new long[DEPTH];
long[] quantities = new long[DEPTH];

Project Valhalla's value classes aim to give flat layouts without the manual work; as of JDK 27 they are available only in early-access builds.

Agrona and Chronicle: what each library provides

Agrona, from the team behind Aeron and SBE, collects the building blocks: direct and atomic buffers over on-heap and off-heap memory, primitive-keyed maps and sets with open addressing, lock-less queues, off-heap ring and broadcast buffers for communication between processes, an agent framework for duty-cycle threads and a timer wheel.

Chronicle Software's Chronicle Queue is a persisted, broker-less queue over memory-mapped files, so every message is journaled as a side effect of being passed and the heap stays small. The vendor states that it passes messages between JVMs on one machine in under a microsecond; replication between machines is part of its commercial edition.

Both libraries give up familiar APIs for control over memory layout and allocation, which pays off on a hot path and rarely across a whole application.

Network I/O and binary serialization

Netty, epoll and batching

Most low-latency Java services that speak TCP run on Netty, directly or through a framework built on it. Three settings matter most. On Linux, the native epoll transport produces less garbage than the NIO transport and exposes Linux-specific socket options, and since Netty 4.2, released in April 2025, the io_uring transport is a fully supported module too (Netty 4.2 migration guide). TCP_NODELAY turns off Nagle's algorithm, which holds small writes back while it waits for an acknowledgement. And nothing that blocks may run on an event-loop thread, because every connection on that loop waits behind it.

EventLoopGroup group = new MultiThreadIoEventLoopGroup(EpollIoHandler.newFactory());   // Netty 4.2
ServerBootstrap bootstrap = new ServerBootstrap()
        .group(group)
        .channel(EpollServerSocketChannel.class)
        .childOption(ChannelOption.TCP_NODELAY, true)
        .childHandler(new ChannelInitializer<SocketChannel>() {
            @Override
            protected void initChannel(SocketChannel ch) {
                ch.pipeline().addLast(new FlushConsolidationHandler(),
                                      new QuoteDecoder(), new QuoteHandler());
            }
        });

Batching works in the opposite direction to Nagle's algorithm. Flushing once per batch of responses instead of once per response cuts system calls, and Netty's FlushConsolidationHandler does it without holding back a lone response. The Disruptor batches the same way on the consumer side: a handler that falls behind receives every available event at once, with endOfBatch set on the last one, and can write them out together.

Kernel bypass, kept in proportion

Every packet that goes through the Linux network stack spends microseconds in the kernel on each side. Kernel bypass removes that cost. AMD's Onload (formerly Solarflare OpenOnload) accelerates ordinary sockets on supported network cards, DPDK hands the card to a user-space driver, and RDMA lets machines read each other's memory.

All of them need specific hardware, privileged configuration and engineers who can debug below the JVM. They help when the network stack is a measurable share of the budget, which is common in trading, where a whole hop is budgeted in microseconds. In a 10 ms service, the kernel's share is noise.

Compact binary formats: SBE

Parsing JSON allocates strings and objects for fields and scans text to find them. Simple Binary Encoding (SBE), maintained by the Aeron project, is a codec designed for messages in low-latency financial applications (SBE). A schema generates flyweight encoders and decoders that read and write fields at fixed offsets directly in a buffer, so decoding allocates nothing.

// SBE-generated flyweights: no objects created per message
quoteDecoder.wrapAndApplyHeader(buffer, offset, headerDecoder);
long instrument = quoteDecoder.instrumentId();
long price = quoteDecoder.price();

SBE messages are not human-readable, schema evolution follows stricter rules than JSON or Protocol Buffers (new fields go at the end of a block, with a version number), and variable-length fields must be read in schema order. For an internal hot path that is usually acceptable. A common split is JSON at the public edge and a binary format inside.

OS and JVM tuning for predictable latency

Once the code is clean, the remaining jitter comes from the machine. These settings are the usual starting point on Linux:

• Huge pages: 2 MB pages instead of 4 KB cut TLB misses on a large heap. -XX:+UseTransparentHugePages works when the kernel's THP mode is madvise; -XX:+UseLargePages uses pages reserved in advance through hugetlbfs, which avoids the stalls of THP's background compaction.
• -XX:+AlwaysPreTouch faults in every heap page at startup, so the first GC cycles under load do not pay for page faults.
• On a two-socket server, memory attached to the other socket is slower to reach. Binding a latency-critical JVM to one socket with numactl --cpunodebind=0 --membind=0 keeps every access local; -XX:+UseNUMA makes the collector NUMA-aware when a JVM has to span sockets.
• Deep C-states and frequency scaling save power by putting idle cores to sleep, and waking them costs microseconds. Latency-critical hosts run the performance governor with deep C-states disabled.
• No swap for the JVM's memory, and no other busy processes on its cores.

Safepoints and time-to-safepoint

Some JVM operations, including parts of GC cycles, deoptimization and class redefinition, need every Java thread stopped at a safepoint. The pause the application sees is the operation plus the time to safepoint: how long the slowest thread takes to reach a safepoint poll. One thread that runs a long stretch of code without a poll holds every other thread at the barrier.

Since JDK 10, the C2 compiler's loop strip mining keeps polls in long counted loops at low cost, so the classic culprit is rarer. What remains is mostly large array copies and page faults while touching memory-mapped files or swapped-out memory. Biased locking, which needed safepoints to revoke biases, was disabled in JDK 15 (JEP 374). Unified logging shows both parts of every safepoint:

-Xlog:safepoint:file=safepoint.log:uptime
# each line: the operation, "Reaching safepoint" (time to safepoint), "At safepoint", "Total"

A "Reaching safepoint" value that is large compared with "At safepoint" points to time to safepoint rather than the operation itself. JFR records the same data in jdk.SafepointBegin and jdk.SafepointStateSynchronization events.

Finding latency outliers with JFR

JDK Flight Recorder runs in production at low overhead and records the events that explain a slow request: GC pauses and phases, safepoints, lock contention (jdk.JavaMonitorEnter, jdk.ThreadPark), socket reads and writes, allocation stalls and deoptimizations. A continuous recording keeps the recent past and can be dumped right after a latency alert:

java -XX:StartFlightRecording:maxage=30m,settings=profile -jar quote-service.jar

# after an alert
jcmd <pid> JFR.dump filename=spike.jfr
jfr view gc-pauses spike.jfr
jfr view safepoints spike.jfr
jfr view contention-by-site spike.jfr

Java 25 added two features that help here. Method timing and tracing (JEP 520) times or traces chosen methods through a filter, with no code change: -XX:StartFlightRecording:method-timing=com.example.PricingEngine::quote, then jfr view method-timing. CPU-time profiling (JEP 509, experimental and Linux only) samples threads by the CPU time they consume, in jdk.CPUTimeSample events. For wall-clock profiles that include time spent waiting, async-profiler's wall mode complements JFR; our guide to Java profilers compares the tools.

What Java 21 to 27 changed for low latency

Recent releases removed several reasons low-latency teams used to stay on older JDKs. Java 25 is the current long-term support release; JDK 26 and JDK 27, released in March and September 2026, are feature releases. Our post on how Java enables high performance covers the wider picture of what makes the JVM fast.

Low-latency Java checklist

These are the checks we work through, in this order, when a Java service misses its latency target:

1. Write the target as a percentile, a load and a measurement point, then split it into a budget per stage with headroom.
2. Remove network and disk hops from the request path before tuning anything inside the JVM.
3. Measure with an open-model load generator (wrk2 -R, Gatling open injection, k6 constant-arrival-rate) and record into HdrHistogram. Never average percentiles.
4. Look for platform stalls with jHiccup and JFR: GC pauses, safepoints, allocation stalls.
5. Choose the collector for the budget: G1 for tens of milliseconds, generational ZGC or Shenandoah below that. Fix the heap size and pre-touch it.
6. Bring the hot path to zero allocation, verified with JMH -prof gc or JFR allocation samples.
7. Warm up with representative traffic before taking load, and build an AOT cache on JDK 25 or later.
8. Give each piece of hot mutable state one writing thread, and pass messages through a ring buffer rather than a locked queue.
9. Spin only on dedicated cores, never under a CPU quota.
10. Replace boxed collections on the hot path with primitive ones, and pad counters that different threads write.
11. Use a binary codec inside the system, with TCP_NODELAY and flush batching at the edge.
12. Tune the host last: huge pages, NUMA binding, the CPU governor, and isolated cores where the budget is in microseconds.

The order matters because the early items find the large wins: in the metasearch engine, moving the data into memory did more for response time than any JVM setting could have. If a Java service of yours meets its median but misses its p99, the first step of our Performance Rescue measures that tail correctly and finds what it is waiting for, and the fix is verified under the same load.