Monitoring and Maintenance Best Practices for Using Proxmox with FlashArray over NVMe-TCP

This page provides the monitoring and maintenance best practices for using Proxmox with flasharray over NVMe-TCP.

Monitoring NVMe Device Health

Note: FlashArray will not return any information back with the SMART Data request. This would apply to local devices.

# Check SMART data
nvme smart-log /dev/nvme0n1

# Key metrics to monitor:
# - Temperature
# - Available Spare
# - Available Spare Threshold
# - Percentage Used
# - Critical Warning flags

Monitoring Path Health

# Check all paths status
nvme list-subsys

# Expected output shows all paths as "live"
# Example:
# nvme-subsys0 - NQN=nqn.2024...
#  +- nvme0 tcp traddr=10.100.1.10 ... live
#  +- nvme1 tcp traddr=10.100.1.11 ... live
#  +- nvme2 tcp traddr=10.100.2.10 ... live
#  +- nvme3 tcp traddr=10.100.2.11 ... live

Monitoring Network Health

# Monitor interface errors
ip -s link show ens1f0

# Check for packet loss
ethtool -S ens1f0 | grep -i error

# Monitor bandwidth utilization
iftop -i ens1f0

Automated Monitoring Script

#!/bin/bash
# /usr/local/bin/nvme-health-check.sh

# Check NVMe subsystem status
SUBSYS_STATUS=$(nvme list-subsys 2>&1)
DEAD_PATHS=$(echo "$SUBSYS_STATUS" | grep -c "dead")

if [ "$DEAD_PATHS" -gt 0 ]; then
    echo "WARNING: $DEAD_PATHS dead NVMe paths detected"
    echo "$SUBSYS_STATUS"
    # Send alert (email, webhook, etc.)
fi

# Check SMART health
for dev in /dev/nvme*n1; do
    CRITICAL=$(nvme smart-log "$dev" | grep "critical_warning" | awk '{print $3}')
    if [ "$CRITICAL" != "0" ]; then
        echo "CRITICAL: NVMe device $dev has critical warning"
        nvme smart-log "$dev"
        # Send alert
    fi
done

# Check network interface status
for iface in ens1f0 ens1f1; do
    if ! ip link show "$iface" | grep -q "state UP"; then
        echo "ERROR: Interface $iface is down"
        # Send alert
    fi
done

To schedule with cron:

# Run every 5 minutes
*/5 * * * * /usr/local/bin/nvme-health-check.sh