Cisco has a published Bug outlining the issue, CSCvo09082, which can be referred to as needed. With the default queue depth with the nfnic driver set to 32, that in turn limits all Maximum Device Queue Depth and DSRNO (Disk Schedule Number Requests Outstanding) to 32. This would apply to device backings for VMFS Datastores, RDMs and Protocol Endpoints. This has primarily been an issue when using vSphere Virtual Volumes as the default device queue depth for a Protocol Endpoint should be 128. This could also affect any high workload and IOPs VMs that need higher queue depths across the board.
Cisco released an updated version of the nfnic driver in 4.0.0.35 to adjust "lun_queue_depth_per_path". This is not the default nfnic driver for ESXi 6.7 U2 and under. This means that this driver will need to be manually installed until a Cisco image or an image from VMware are released with an updated nfnic driver.
Everpure does recommend using the nfnic driver version 4.0.0.52 or higher if the hosts are running ESXi 6.7 or higher.
This can be downloaded from VMware here.
You can get the updated nfnic driver from Cisco's UCS Blade Software page (the 4.0.4 release which can be found here) and from VMware directly as well here.
In this article the method of updating the nfnic driver, setting the nfnic "lun_queue_depth_per_path" parameter and updating the Protocol Endpoint's PESRNO (Protocol Endpoint Schedule Number Requests Outstanding) will be outlined and demonstrated.