Initiator Performance Enhancement

Increasing the performance of an EC2 instance depends on several factors, mostly it depends on the instance type, hence the number of vCPUs that it has. Therefore, there is a baseline to the amount of traffic the instance can utilize. This section covers what are the other options can be leveraged to enhance the network performance.

Jumbo Frames

Increasing the instance network MTU size to 9000 bytes (also commonly knows as jumbo frames) improves the network efficiency by allowing bigger payloads to be sent over the network in fewer packets while maintaining the same network overhead. Although enabling jumbo frame allows for higher network throughput, it is not a straight forward decision to make and requires proper investigation into your AWS network environment and how your workload is handling I/O requests.

In general, It is not recommended to enable jumbo frames on a network interface that sends traffic over an internet gateway therefore leaves your VPC, since this might lead to packets fragmentation by an intermediate or receiver systems. The same rule applies to sending traffic over an inter-region VPC peering connection or traffic over VPN connections.

To prevent slower performance and packets dropping due to MTU size mismatching, follow the best practices below.

iSCSI Sessions

It is important to note that AWS has bandwidth limits on each TCP connection. A single TCP connection is only capable of 5 Gb networking in AWS. Since a single iSCSI session equates to a single TCP connection, each iSCSI session is also limited on throughput. For applications that need higher throughput, it is advised to increase the number of iSCSI sessions on the EC2 host. If you are looking to maximize throughput from a given EC2, the number of iSCSI sessions will vary depending on the size of the EC2 instance. The Windows and Linux examples above already includes steps to set these values.

The approximate guidance is to have 2 iSCSI sessions for each EC2 "xlarge" size. You can always increase the number of sessions beyond approximate guidance (up to 32 iSCSI per controller) if needed but we recommend NOT going over 16 iSCSI sessions. Especially with larger block sizes, higher number of iSCSI sessions may lead to increase in latency without providing benefits in the form of higher IOPS and throughput performance.

For example, an application running on an EC2 size of:

- c5.2xlarge would have 4 iSCSI sessions to each PSC Dedicated controller.

- m5.4xlarge would have 8 iSCSI sessions to each PSC Dedicated controller.

- r5.8xlarge would have 16 iSCSI sessions to each PSC Dedicated controller.

- r5n.16xlarge would have 16 iSCSI sessions to each PSC Dedicated controller.

The guidance under Host Management | PSC Dedicated for AWS provide approximate values and can be increased, up to 32 iSCSI sessions. We strongly recommend not going over 16 iSCSI sessions per controller.

MPIO Settings

I/O should only flow to the primary controller. Ensure you followed the steps to appropriately set MPIO parameters for Setup Windows iSCSI for use with Everpure Cloud Dedication and Setup Linux iSCSI foruse with Everpure Cloud Dedication hosts to ensure IO's are sent to the primary controller.

To confirm, run I/O on your host and run the following command on your PSC Dedicated instance:

purehost monitor --balance --interval 3

pureuser@CBS> purehost monitor --balance --interval 3
Name       Time                     Initiator WWN  Initiator IQN                        Initiator NQN  Target       Target WWN  Failover  I/O Count  I/O Relative to Max
Linux2AMI  2019-08-26 10:31:32 PDT  -              iqn.1994-05.com.redhat:b9ddc64322ef  -              (primary)    -           -         1626       100%
                                                   iqn.1994-05.com.redhat:b9ddc64322ef                 (secondary)              -         0          0%
                                                  

Network Bandwidth Limits

Keep in mind that each Amazon EC2 instance type has network bandwidth limits. See Amazon EC2 instance types. When running performance load testing with a PSC Dedicated instance, ensure the Amazon EC2 instance that the application is running on has enough network bandwidth. For example, a PSC Dedicated V20A-R1 instance uses C5n.18xlarge instances for the controllers which has up to 100 Gb of bandwidth. Therefore the application host should use a single Amazon EC2 instance with matching network limits (ex: 1 x C5n.18xlarge), or multiple Amazon EC2 instances with networks that add up to the 100 Gb limit (ex: 4 x r5n.8xlarge). This ensures that the application is not the bottle neck.

Elastic Network Adapter (ENA)

Leveraging Elastic Network Adapter (ENA) on EC2 instances can deliver high-performance networking capabilities on supported instance types. Enhanced networking uses single root I/O virtualization (SR-IOV) which resulting in near bare metal performance.