stratoflow

Latency numbers every Java developer should know

We care a lot about (low) latency at Stratoflow and this is not only when building high performance computing processes but for any type of applications really.

Why does latency matter?

We could observe a major shift in hard availability and its cost/performance ratio in the last 10 years, and while this also relates to processors the real impact is made by RAM memory and SSD storage devices.

Memory units became cheap, large and fast, and you can order real memory monsters with terabytes of RAM with a few clicks right now. Well, your credit card limit applies but joke asides – just see Dell PowerEdge R940 (6 TB RAM) or HPE Integrity MC990 X Server (48 TB RAM).

Yes, this is 48 TB of DDR4 RAM in one physical box at your service.

SSD storage devices are amazing at their speed, typically at least 20x faster than good old spinning disks. You can easily extend your desktop or laptop with Samsung 850 PRO disks or go with specialized SSD cards for your servers.

latency number

What does it mean to us, Java devs?

Just take a look at this brilliant summary of latency times required to transfer data between different physical units of your computer system: Latency Numbers Every Programmer Should Know:

Latency Comparison Numbers
 --------------------------
 L1 cache reference 0.5 ns
 Branch mispredict 5 ns
 L2 cache reference 7 ns 14x L1 cache
 Mutex lock/unlock 25 ns
 Main memory reference 100 ns 20x L2 cache, 200x L1 cache
 Compress 1K bytes with Zippy 3,000 ns 3 us
 Send 1K bytes over 1 Gbps network 10,000 ns 10 us
 Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD
 Read 1 MB sequentially from memory 250,000 ns 250 us
 Round trip within same datacenter 500,000 ns 500 us
 Read 1 MB sequentially from SSD* 1,000,000 ns 1,000 us 1 ms ~1GB/sec SSD, 4X memory
 Disk seek 10,000,000 ns 10,000 us 10 ms 20x datacenter roundtrip
 Read 1 MB sequentially from disk 20,000,000 ns 20,000 us 20 ms 80x memory, 20X SSD
 Send packet CA->Netherlands->CA 150,000,000 ns 150,000 us 150 ms

Notes
 -----
 1 ns = 10^-9 seconds
 1 us = 10^-6 seconds = 1,000 ns
 1 ms = 10^-3 seconds = 1,000 us = 1,000,000 ns

Credit
 ------
 By Jeff Dean: http://research.google.com/people/jeff/
 Originally by Peter Norvig: http://norvig.com/21-days.html#answers

Yes, main memory is 4x faster than SSD, and SSD is 20x faster than a spinning hard drive.

[Read also: How to Build Low Latency Java Applications]

So what are the quick lessons for us?

  1. “Your data fits in RAM. Yes, it does.” – see RAM is the New SSD.
  2. Design your system to store all data in memory or cache aggressively otherwise. Observe and eliminate I/O hits.
  3. Process data in memory and leave the results there.
  4. Worried about persistency and transactional control? Try to restructure your processing to write data asynchronously into a persistent storage device and measure your throughput/latency twice – it may be that you can just reprocess whole batch once again in memory in case of any error.
  5. Really not enough memory to store your data? Or more CPU power needed to crunch your data? Look at distributed cache and grid-computing platforms like Hazelcast, Ignite or Coherence

Check out all of our Java software development services here.

[Follow up reading: Java Clean Code: How to Write It? Best Practices From Experts]

Testimonials

The developed software product was built from scratch with solid quality. We have had a long-term engagement with Stratoflow for nearly 10 years. We look at them as partners, rather than contractors. I'm impressed by their team culture and cross-team support.

Nathan Pesin

CTO, Legerity Financials

Stratoflow was a great partner, challenging as well as supporting our customer projects for the best outcome. They have a great pool of talent within the business - all very capability technologists, as well as being business-savvy and suitable for consultancy engagements.

Chris Goodall

Managing Consultant, CG Consultancy (UK) Limited

The bespoke metal exchange platform works great, it is easily accessible and richly functional. Stratoflow managed deadlines capably, meticulously documented their progress, and delivered a complex project at an affordable cost.

Bartlomiej Knichnicki

Vice Chairman, Supervisory Board

We are very pleased with our partnership with Stratoflow and, as we continue to grow, we expect to increase the numbers of developers that work with us on our projects. They have proven to be very skilled and flexible. They're extremely reliable, and they have a very good company culture of their own, which gives them a real edge compared to other providers that serve more as production shops rather than thought partners and creative problem solvers.

Andrew Kennedy

Founder & Managing Director, Tier 2 Consulting

Stratoflow successfully customized the system according to the specific functionalities and without bugs reported. The team was commended for their adaptability in the work process and for their responsiveness.

Joshua Blavins

Tech PM, Digital Agency

The features implemented have received overwhelmingly positive feedback from end-users. Stratoflow has an incredible technical expertise and a high degree of flexibility when it comes to changing project requirements.

Adam Hill

Chief Technology Officer, Legerity

They have impressively good knowledge of AI issues. Very responsive to any amendments and findings. Very good communication. We received a finished project which could be implemented into production shortly after testing.

CO-Founder & CTO

Circular Fashion Company

They provided superb service with seamless communication and a highly professional, technical approach. The team displays impressive technical expertise and are willing to share information and engage in constructive feedback.

Filip Stachnik

Operations Manager, Otwarte Klatki (part of Anima International)

They're very skilled technically and are also able to see the bigger picture. Stratoflow can actually think about solutions, not just the technical task at hand, which they've been assigned.

Arnd Jan Prause

Chief Operating Officer, musQueteer

Stratoflow delivered the website successfully within the timeframe and budget. They assured that the output met the set requirements. Overall, the team's performance was excellent and recommended for their exceptional technical business expertise. They've been able to deliver all of their work on time and within budget, which has been very impressive.

Lars Andersen

Founder & CEO, My Nametags

Travel sector rebound after the pandemic is complete. We have fantastic global coverage of travel data distribution due to mutual agreements and data exchange between aggregators. Competition for the best price of limited resources degradates margins.

How to win? Provide personalized experience and build your own products in the front-office. The missing bits: a traveller golden record collecting past activities and a AI/ML recommendation technology.

Michał Głomba

CEO at Stratoflow