# Linux 使用SR-IOV时PF VF重命名解决方法 By [likizju](https://paragraph.com/@likizju) · 2021-11-11 --- #Linux 使用SR-IOV时PF VF重命名解决方法 \[TOC\] ##背景 DELL R620 一块板载Broadcom网卡（双千兆，双万兆光口） 2块外接的INTEL PCIE双光口万兆网卡 root@hk4-31:~# lspci -nn|grep -i net 01:00.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM57800 1/10 Gigabit Ethernet [14e4:168a] (rev 10) 01:00.1 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM57800 1/10 Gigabit Ethernet [14e4:168a] (rev 10) 01:00.2 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM57800 1/10 Gigabit Ethernet [14e4:168a] (rev 10) 01:00.3 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM57800 1/10 Gigabit Ethernet [14e4:168a] (rev 10) 04:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) 04:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) 41:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) 41:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) 现在想使用其中一块外接INTEL网卡做VF给KVM虚拟机使用，要解决： * 只启用指定网卡的SR-IOV * VF产生后会和系统的网卡顺序混淆，使用时非常难找，因此需要重命名 ##VT-d及VF配置 * 安装sysfsutils,并且指定外接其中一块网卡的2个端口才启用VF： bus/pci/devices/0000:04:00.0/sriov_numvfs = 8 bus/pci/devices/0000:04:00.1/sriov_numvfs = 8 * 启用INTEL的VT-d，除了BIOS启用外还要修改GRUB启动参数 GRUB_CMDLINE_LINUX="intel_iommu=on" 后更新grub: update-grub 重启后即可以看到： 01:00.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM57800 1/10 Gigabit Ethernet [14e4:168a] (rev 10) 01:00.1 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM57800 1/10 Gigabit Ethernet [14e4:168a] (rev 10) 01:00.2 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM57800 1/10 Gigabit Ethernet [14e4:168a] (rev 10) 01:00.3 Ethernet controller [0200]: Broadcom Corporation NetXtreme II BCM57800 1/10 Gigabit Ethernet [14e4:168a] (rev 10) 04:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) 04:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) 04:10.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:10.1 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:10.2 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:10.3 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:10.4 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:10.5 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:10.6 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:10.7 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:11.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:11.1 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:11.2 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:11.3 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:11.4 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:11.5 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:11.6 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 04:11.7 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) 41:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) 41:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) ##UDEV修改解决网卡重命名问题 脚本，用于查看系统网卡及udev: #!/bin/sh nets=$(ifconfig -a|grep -E 'eth|vf|pf' |awk '{print $1}') for i in $nets do driver=$(ethtool -i $i|grep driver) bus=$(ethtool -i $i|grep bus-in) link=$(ethtool $i | grep 'Link det') speed=$(ethtool $i|grep 'Speed') echo "$i $driver $bus $link $speed" done | column -t regexp='looking at parent device.*\/devices\/pci[0-9]+:[0-9]+\/[0-9]+:[0-9]+:[0-9]+.[0-9]+\/[0-9]+:[0-9]+:[0-9]+\.[0-9]+' nics=$(ls -d /sys/class/net/eth* /sys/class/net/pf* /sys/class/net/vf*) for nic in $nics do udevadm info -a $nic | grep -A3 -E "$regexp"|tail -n3|tr '\n' '\t' i=$(echo $nic | cut -d '/' -f5) echo "NAME=\"$i\"" done 查看目前的结果： eth0 driver: bnx2x bus-info: 0000:01:00.0 Speed: 10000Mb/s eth1 driver: bnx2x bus-info: 0000:01:00.1 Speed: 10000Mb/s eth3 driver: bnx2x bus-info: 0000:01:00.2 Speed: 1000Mb/s eth4 driver: bnx2x bus-info: 0000:01:00.3 Speed: 1000Mb/s eth8 driver: ixgbevf bus-info: 0000:04:11.0 Speed: Unknown! eth9 driver: ixgbevf bus-info: 0000:04:10.2 Speed: Unknown! eth10 driver: ixgbevf bus-info: 0000:04:10.4 Speed: Unknown! eth11 driver: ixgbevf bus-info: 0000:04:10.6 Speed: Unknown! eth12 driver: ixgbevf bus-info: 0000:04:10.0 Speed: Unknown! eth13 driver: ixgbevf bus-info: 0000:04:11.2 Speed: Unknown! eth14 driver: ixgbevf bus-info: 0000:04:11.4 Speed: Unknown! eth15 driver: ixgbevf bus-info: 0000:04:11.6 Speed: Unknown! eth16 driver: ixgbevf bus-info: 0000:04:10.1 Speed: Unknown! eth17 driver: ixgbevf bus-info: 0000:04:10.3 Speed: Unknown! eth18 driver: ixgbevf bus-info: 0000:04:10.5 Speed: Unknown! eth19 driver: ixgbevf bus-info: 0000:04:10.7 Speed: Unknown! eth20 driver: ixgbevf bus-info: 0000:04:11.1 Speed: Unknown! eth21 driver: ixgbevf bus-info: 0000:04:11.3 Speed: Unknown! eth22 driver: ixgbevf bus-info: 0000:04:11.5 Speed: Unknown! eth23 driver: ixgbevf bus-info: 0000:04:11.7 Speed: Unknown! KERNELS=="0000:01:00.0" SUBSYSTEMS=="pci" DRIVERS=="bnx2x" NAME="eth0" KERNELS=="0000:01:00.1" SUBSYSTEMS=="pci" DRIVERS=="bnx2x" NAME="eth1" KERNELS=="0000:04:10.4" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth10" KERNELS=="0000:04:10.6" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth11" KERNELS=="0000:04:11.2" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth13" KERNELS=="0000:04:11.4" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth14" KERNELS=="0000:04:11.6" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth15" KERNELS=="0000:04:10.1" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth16" KERNELS=="0000:04:10.3" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth17" KERNELS=="0000:04:10.5" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth18" KERNELS=="0000:04:10.7" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth19" KERNELS=="0000:04:00.0" SUBSYSTEMS=="pci" DRIVERS=="ixgbe" NAME="eth2" KERNELS=="0000:04:11.1" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth20" KERNELS=="0000:04:11.3" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth21" KERNELS=="0000:04:11.5" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth22" KERNELS=="0000:04:11.7" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth23" KERNELS=="0000:01:00.2" SUBSYSTEMS=="pci" DRIVERS=="bnx2x" NAME="eth3" KERNELS=="0000:01:00.3" SUBSYSTEMS=="pci" DRIVERS=="bnx2x" NAME="eth4" KERNELS=="0000:04:00.1" SUBSYSTEMS=="pci" DRIVERS=="ixgbe" NAME="eth5" KERNELS=="0000:41:00.0" SUBSYSTEMS=="pci" DRIVERS=="ixgbe" NAME="eth6" KERNELS=="0000:41:00.1" SUBSYSTEMS=="pci" DRIVERS=="ixgbe" NAME="eth7" KERNELS=="0000:04:10.2" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="eth9" 由于VF的MAC地址有不确定性，所以这里我们选择以PCI BUS ID来作为命名依据： `vi /etc/udev/rules.d/99-pfvf-net.rules` 并且去掉默认生成的net.rules里这几个网卡的内容 KERNELS=="0000:04:00.0" SUBSYSTEMS=="pci" DRIVERS=="ixgbe" NAME="pf-hk" KERNELS=="0000:04:00.1" SUBSYSTEMS=="pci" DRIVERS=="ixgbe" NAME="pf-cmcc" KERNELS=="0000:04:10.0" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf0-hk" KERNELS=="0000:04:10.2" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf1-hk" KERNELS=="0000:04:10.4" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf2-hk" KERNELS=="0000:04:10.6" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf3-hk" KERNELS=="0000:04:11.0" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf4-hk" KERNELS=="0000:04:11.2" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf5-hk" KERNELS=="0000:04:11.4" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf6-hk KERNELS=="0000:04:11.6" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf7-hk" KERNELS=="0000:04:10.1" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf0-cmcc" KERNELS=="0000:04:10.3" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf1-cmcc" KERNELS=="0000:04:10.5" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf2-cmcc" KERNELS=="0000:04:10.7" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf3-cmcc" KERNELS=="0000:04:11.1" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf4-cmcc" KERNELS=="0000:04:11.3" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf5-cmcc" KERNELS=="0000:04:11.5" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf6-cmcc" KERNELS=="0000:04:11.7" SUBSYSTEMS=="pci" DRIVERS=="ixgbevf" NAME="vf7-cmcc" 需要注意的是， vf的pci id并不是连续的， 在2个PF的情况下，这些VF交叉出现 现在让命名生效，先卸载PF和VF的驱动： modprobe -r ixgbe modprobe -r ixgbevf 更新 udev设备名: udevadm trigger --action=add 然后重新加载驱动，即可以看到想要的顺序了： eth0 driver: bnx2x bus-info: 0000:01:00.0 Link detected: yes Speed: 10000Mb/s eth1 driver: bnx2x bus-info: 0000:01:00.1 Link detected: yes Speed: 10000Mb/s eth3 driver: bnx2x bus-info: 0000:01:00.2 Link detected: yes Speed: 1000Mb/s eth4 driver: bnx2x bus-info: 0000:01:00.3 Link detected: yes Speed: 1000Mb/s eth6 driver: ixgbe bus-info: 0000:41:00.0 Link detected: yes Speed: 10000Mb/s eth7 driver: ixgbe bus-info: 0000:41:00.1 Link detected: yes Speed: 10000Mb/s pf-cmcc driver: ixgbe bus-info: 0000:04:00.1 Link detected: no Speed: 10000Mb/s pf-hk driver: ixgbe bus-info: 0000:04:00.0 Link detected: no Speed: 10000Mb/s vf0-cmcc driver: ixgbevf bus-info: 0000:04:10.1 Link detected: no Speed: Unknown! vf0-hk driver: ixgbevf bus-info: 0000:04:10.0 Link detected: no Speed: Unknown! vf1-cmcc driver: ixgbevf bus-info: 0000:04:10.3 Link detected: no Speed: Unknown! vf1-hk driver: ixgbevf bus-info: 0000:04:10.2 Link detected: no Speed: Unknown! vf2-cmcc driver: ixgbevf bus-info: 0000:04:10.5 Link detected: no Speed: Unknown! vf2-hk driver: ixgbevf bus-info: 0000:04:10.4 Link detected: no Speed: Unknown! vf3-cmcc driver: ixgbevf bus-info: 0000:04:10.7 Link detected: no Speed: Unknown! vf3-hk driver: ixgbevf bus-info: 0000:04:10.6 Link detected: no Speed: Unknown! vf4-cmcc driver: ixgbevf bus-info: 0000:04:11.1 Link detected: no Speed: Unknown! vf4-hk driver: ixgbevf bus-info: 0000:04:11.0 Link detected: no Speed: Unknown! vf5-cmcc driver: ixgbevf bus-info: 0000:04:11.3 Link detected: no Speed: Unknown! vf5-hk driver: ixgbevf bus-info: 0000:04:11.2 Link detected: no Speed: Unknown! vf6-cmcc driver: ixgbevf bus-info: 0000:04:11.5 Link detected: no Speed: Unknown! vf6-hk driver: ixgbevf bus-info: 0000:04:11.4 Link detected: no Speed: Unknown! vf7-cmcc driver: ixgbevf bus-info: 0000:04:11.7 Link detected: no Speed: Unknown! vf7-hk driver: ixgbevf bus-info: 0000:04:11.6 Link detected: no Speed: Unknown! ##注意事项 别忘记在`/etc/network/interfaces`里添加pf自启动，不需要配置ip，但是如果PF是DOWN的，VF在虚拟机里将不能使用 auto pf-cmcc iface pf-cmcc inet manual up ip link set pf-cmcc up auto pf-hk iface pf-hk inet manual up ip link set pf-hk up --- *Originally published on [likizju](https://paragraph.com/@likizju/linux-sr-iov-pf-vf)*