Latency numbers every Java developer should know

We care a lot about (low) latency at Stratoflow and this is not only when building high performance computing processes but for any type of applications really.

Why does latency matter?

We could observe a major shift in hard availability and its cost/performance ratio in the last 10 years, and while this also relates to processors the real impact is made by RAM memory and SSD storage devices.

Memory units became cheap, large and fast, and you can order real memory monsters with terabytes of RAM with a few clicks right now. Well, your credit card limit applies but joke asides – just see Dell PowerEdge R940 (6 TB RAM) or HPE Integrity MC990 X Server (48 TB RAM).

Yes, this is 48 TB of DDR4 RAM in one physical box at your service.

SSD storage devices are amazing at their speed, typically at least 20x faster than good old spinning disks. You can easily extend your desktop or laptop with Samsung 850 PRO disks or go with specialised SSD cards for your servers.

 

What does it mean to us, Java devs?

Just take a look at this brilliant summary of latency times required to transfer data between different physical units of your computer system: Latency Numbers Every Programmer Should Know:

Latency Comparison Numbers
 --------------------------
 L1 cache reference 0.5 ns
 Branch mispredict 5 ns
 L2 cache reference 7 ns 14x L1 cache
 Mutex lock/unlock 25 ns
 Main memory reference 100 ns 20x L2 cache, 200x L1 cache
 Compress 1K bytes with Zippy 3,000 ns 3 us
 Send 1K bytes over 1 Gbps network 10,000 ns 10 us
 Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD
 Read 1 MB sequentially from memory 250,000 ns 250 us
 Round trip within same datacenter 500,000 ns 500 us
 Read 1 MB sequentially from SSD* 1,000,000 ns 1,000 us 1 ms ~1GB/sec SSD, 4X memory
 Disk seek 10,000,000 ns 10,000 us 10 ms 20x datacenter roundtrip
 Read 1 MB sequentially from disk 20,000,000 ns 20,000 us 20 ms 80x memory, 20X SSD
 Send packet CA->Netherlands->CA 150,000,000 ns 150,000 us 150 ms

Notes
 -----
 1 ns = 10^-9 seconds
 1 us = 10^-6 seconds = 1,000 ns
 1 ms = 10^-3 seconds = 1,000 us = 1,000,000 ns

Credit
 ------
 By Jeff Dean: http://research.google.com/people/jeff/
 Originally by Peter Norvig: http://norvig.com/21-days.html#answers

 

Yes, main memory is 4x faster than SSD, and SSD is 20x faster than a spinning hard drive.

 

So what are the quick lessons for us?

  1. “Your data fits in RAM. Yes, it does.” – see RAM is the New SSD.
  2. Design your system to store all data in memory or cache aggressively otherwise. Observe and eliminate I/O hits.
  3. Process data in memory and leave the results there.
  4. Worried about persistency and transactional control? Try to restructure your processing to write data asynchronously into a persistent storage device and measure your throughput/latency twice – it may be that you can just reprocess whole batch once again in memory in case of any error.
  5. Really not enough memory to store your data? Or more CPU power needed to crunch your data? Look at distributed cache and grid-computing platforms like Hazelcast, Ignite or Coherence

Comments (0)

Thanks! Your comment is awaiting moderation.

Testimonials

They have a very good company culture of their own, which gives them a real edge compared to other providers.

CEO

Leading UK system integrator

They're very skilled technically and are also able to see the bigger picture.

Managing Partner

Scalable SaaS for healthcare IoT built on Salesforce platform

They've been consistently able to deliver work on time and within budget.

CTO

High performance SaaS for financial insitutions

We are seriously impressed by the quality and broader picture of anything they do for us.

CEO

Gold trading platform

Speak with an Expert

How can we help? If you would like a member of the Stratoflow team to get in touch, please send us your message and we will contact you shortly.

Contact