Linux Hosts Configuration

Everpure Cloud Dedicated for AWS
Audience
Public
Technology Integrations
Amazon Web Services (AWS)
Source Type
Documentation

We need to make the following changes which are specified to the cloud environment. Other configuration settings are the same as with FlashArray.

Load nvme-tcp kernel module

Note: Run following commands as root user or use sudo.
Make sure you load NVMe-TCP module on the host:

modprobe nvme-tcp

#To verify if nvme-tcp module is loaded
lsmod | grep nvme
nvme_tcp               49152  0
nvme_fabrics           36864  1 nvme_tcp
nvme_core             221184  2 nvme_tcp,nvme_fabrics
nvme_common            24576  1 nvme_core
t10_pi                 24576  2 sd_mod,nvme_core
If NVMe-TCP module is not loaded automatically after host reboot:

tee /etc/modules-load.d/nvme-tcp.conf <<<'nvme_tcp'

NVMe I/O Timeout

For AWS, we recommend setting the NVMe I/O timeout to 30 seconds. Follow OS dependent steps to set the NVMe I/O timeout. On RHEL hosts, you can set the NVMe I/O timeout by:

Ephemeral configuration 
# NOTE: this parameter only applies to newly created NVMe controllers
# existing NVMe controllers will need to be deleted and re-created (reset is insufficient)
modprobe nvme_core
echo 30 > /sys/module/nvme_core/parameters/io_timeout
Persistent configuration 

echo options nvme_core io_timeout=30 > /etc/modprobe.d/nvme-io-timeout.conf
dracut -f
reboot

Multipath.conf

This is the recommended contents of the device mapper multipath configuration file /etc/multipath.conf for NVMe/TCP on PAWS:

defaults {
       polling_interval 10
       user_friendly_names no
       find_multipaths yes
}

devices {
        device {
               vendor                "NVME"
               product               "Pure Storage FlashArray"
               path_selector         "queue-length 0"
               path_grouping_policy  group_by_prio
               fast_io_fail_tmo      10
               no_path_retry         queue
               prio                  ana
               failback			immediate
       }
       device {
               vendor                "PURE"
               product               "FlashArray"
               path_selector         "queue-length 0"
               path_grouping_policy  group_by_prio
               path_checker          tur
               fast_io_fail_tmo      10
               no_path_retry         queue
               hardware_handler      "1 alua"
               prio                  alua
               failback              immediate
       }
}

Consult man 5 multipath.conf and/or the RHEL Documentation before making modifications to the configuration.

After modifying the multipath.conf, make sure to restart multipathd. To restart the device mapper multipath service, run following command:
systemctl restart multipathd
You can run the following command to check the status of multipathd
systemctl status multipathd
To ensure it starts after a reboot, run the following command:
systemctl enable multipathd

Disable Native NVMe multipath if enabled

To check if Native NVMe multipath is enabled or not, execute following command: 

cat /sys/module/nvme_core/parameters/multipath
Y
# The multipath parameter is Y ("yes") -> it's enabled

cat /sys/module/nvme_core/parameters/multipath
N
# The multipath parameter will is N ("no") -> native NVMe multipath is disabled

Please follow your OS dependent steps to disable native multipathing. Below are examples how to disable native NVMe multipath on common Linux distributions (reboot after the changes to take effect).

On RHEL 8.x hosts, you can disable native multipath by using following command:

echo options nvme_core multipath=N > /etc/modprobe.d/native-multipath.conf
dracut -f
On RHEL 9.x host, you can disable native multipath by using following command:
grubby --update-kernel=ALL --args="nvme_core.multipath=N"
For other distributions (e.g. Ubuntu 22.04), an alternative way to disable native multipath is modifying kernel parameters in grub.cfg:
GRUB_CMDLINE_LINUX_DEFAULT="nvme_core.io_timeout=30 nvme_core.multipath=N"