Latency numbers every Java developer should know
We care a lot about (low) latency at Stratoflow and this is not only when building high performance computing processes but for any type of applications really.
Why does latency matter?
We could observe a major shift in hard availability and its cost/performance ratio in the last 10 years, and while this also relates to processors the real impact is made by RAM memory and SSD storage devices.
Memory units became cheap, large and fast, and you can order real memory monsters with terabytes of RAM with a few clicks right now. Well, your credit card limit applies but joke asides – just see Dell PowerEdge R940 (6 TB RAM) or HPE Integrity MC990 X Server (48 TB RAM).
Yes, this is 48 TB of DDR4 RAM in one physical box at your service.
SSD storage devices are amazing at their speed, typically at least 20x faster than good old spinning disks. You can easily extend your desktop or laptop with Samsung 850 PRO disks or go with specialized SSD cards for your servers.
What does it mean to us, Java devs?
Just take a look at this brilliant summary of latency times required to transfer data between different physical units of your computer system: Latency Numbers Every Programmer Should Know:
Latency Comparison Numbers -------------------------- L1 cache reference 0.5 ns Branch mispredict 5 ns L2 cache reference 7 ns 14x L1 cache Mutex lock/unlock 25 ns Main memory reference 100 ns 20x L2 cache, 200x L1 cache Compress 1K bytes with Zippy 3,000 ns 3 us Send 1K bytes over 1 Gbps network 10,000 ns 10 us Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD Read 1 MB sequentially from memory 250,000 ns 250 us Round trip within same datacenter 500,000 ns 500 us Read 1 MB sequentially from SSD* 1,000,000 ns 1,000 us 1 ms ~1GB/sec SSD, 4X memory Disk seek 10,000,000 ns 10,000 us 10 ms 20x datacenter roundtrip Read 1 MB sequentially from disk 20,000,000 ns 20,000 us 20 ms 80x memory, 20X SSD Send packet CA->Netherlands->CA 150,000,000 ns 150,000 us 150 ms Notes ----- 1 ns = 10^-9 seconds 1 us = 10^-6 seconds = 1,000 ns 1 ms = 10^-3 seconds = 1,000 us = 1,000,000 ns Credit ------ By Jeff Dean: http://research.google.com/people/jeff/ Originally by Peter Norvig: http://norvig.com/21-days.html#answers
Yes, main memory is 4x faster than SSD, and SSD is 20x faster than a spinning hard drive.
[Read also: How to Build Low Latency Java Applications]
So what are the quick lessons for us?
- “Your data fits in RAM. Yes, it does.” – see RAM is the New SSD.
- Design your system to store all data in memory or cache aggressively otherwise. Observe and eliminate I/O hits.
- Process data in memory and leave the results there.
- Worried about persistency and transactional control? Try to restructure your processing to write data asynchronously into a persistent storage device and measure your throughput/latency twice – it may be that you can just reprocess whole batch once again in memory in case of any error.
- Really not enough memory to store your data? Or more CPU power needed to crunch your data? Look at distributed cache and grid-computing platforms like Hazelcast, Ignite or Coherence
Check out all of our Java software development services here.
[Follow up reading: Java Clean Code: How to Write It? Best Practices From Experts]
