A 25 second pause in I/O during a single link loss with iSCSI

A question frequently asked is: “Why do I see a pause in I/O to back-end storage when I have multiple paths available and only a single path was taken offline?”

You have taken the time to setup an environment in a redundant and resilient manner and then suddenly a single path is dropped and I/O isn’t serviced for a period of 20-35 seconds.

So let’s dig into this. We will discuss the following things throughout this post:

What are ‘No Operation’ (NOP) PDUs?
What happens if NOPs are not responded to?
What is the iSCSI Recovery Timeout value?
How can I lower the I/O pause if 20-35 seconds is just too high?

NOTE: While we will be referencing VMware vSphere specific values in this post, the concepts are the same for other operating systems. Their parameters may have slightly different names but the functionality is the same.