You cannot change the speed of light and flash storage may notice it

You cannot change the speed of light and flash storage may notice it especially when using data replication between data centers

With flash storage the distance between data center has more impact on the overall response time.

Why?

The speed of light in glass does not change whether you are using nearline, spinning disk or flash storage but the distance matter more when using flash storage

Let’s calculate the latency due to the distance

The speed of light is pretty much given: c0 = 300'000 km/s

 The refraction index (n, for our fiber optic nfo) is defined as the speed of light in vacuum (c0) divided by its speed in a specific medium (cfo).

For fiber optics that's typically in a range of 1.4 to 1.6. So let's take nfo 1.5 as an example.

nfo = c0 / cfo

Rearranging that equation for the speed in fiber optics gives us:

cfo = c0 / nfo = 300'000 km/s / 1.5 = 200'000 km/s

 The reciprocal tells us how long the light travels through a 1 km fiber cable:

tfo = 1 km / cfo = 1 km / 200'000 km/s = 5 * 106 s = 5 µs

As a rule of thumb, a link with a round-trip latency of 1 ms spans approximately 100 km. or seen from another angle a link that spans 100 km has an approximate round-trip time of 1ms (which adds to the latency in synchronous data replication across that distance).

Now let’s see in % the influence of the distance to the overall response time

 We can consider between 10% to 20 % as a noticeable impact, you will reach this for example starting 20 KM distance for a flash storage with 2ms response time

This previous table does not take into account the possible latency that can be added with compression, IPsec , FEC

• Encryption/compression is in the range of 5.5 μsec per node; (BROCADE 6510)
• Forward error correction (FEC) adds 400 ns between E_Ports (enabled by default).
• Latency for locally switched ports is in the range of 700 ns ( Brocade300)
• Brocade extension guide mention: IPsec has been implemented in HW and operates at line rate, adding negligible propagation delay.

Whatever the synchronous technology you are using, you need to give the round-trip time to the frame

How to overcome this

• There are solutions that can spoof the acknowledge.
• For example WAN accelerator (that can be used with FCIP) can locally acknowledge TCP segment to the local device.
• The draw back is that any lost packet within a window requires that the entire window be re transmitted – unless your TCP/IP stack on the server uses selective acknowledgements”

Here is an interesting description of the different replication to use

but you still cannot beat the speed of light in glass,except with hollow-core photonic crystal fibres