ProxmoxVE 6.4-13(PVE) 开启SR-IOV INTEL I350-T4 手把手教程
Single root input/output virtualization(SR-IOV), 是一种先进的虚拟化技术,以网卡为例,它可以在一个物理功能PF(一个网口)上开出多个虚拟功能(虚拟网卡)。
为了令虚拟机与外界的连接高效可靠,往往会采用直通网卡(PCIE Passthrough)的方法,但若是物理机中存在较多的虚拟机,这一方法便不再可行。此外,采取直通网卡的诸多虚拟机之间,并没有网络连接。
SR-IOV针对性的解决了以上两个问题,开启SR-IOV之后,物理网口PF与虚拟网口VFs(可以存在多个)间会建立一个虚拟交换机,这个虚拟交换机具有很好的性能,往往还支持RDMA等高级功能。倘若将VF直通进入虚拟机,这一虚拟机就会与外界、与其他挂在同一PF下的虚拟机,建立高效的连接。这些连接是绕过宿主机内核的,这将带来极好的网络性能。
首先要在主板中打开SR-IOV选项,华擎以及华硕的绝大部分主板都是支持的,b150这种也可能支持,华擎的有些bios内没有选项,但实际上是支持的
查看网是否正确识别出来,lspci -nn|grep Eth
root@AIOserver-Pve:/sys/bus/pci/devices/0000:01:00.0# lspci -nn|grep Eth
00:1f.6 Ethernet controller [0200]: Intel Corporation Device [8086:0d55]
01:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
01:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
01:00.2 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
01:00.3 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
最前面的数字是设备的BDF号
每一个PCIe设备可以只有一个功能(Function),即Fun0。也可以拥有最多8个功能,即多功能设备(Multi-Fun)。不管这个PCIe设备拥有多少个功能,其每一个功能都有一个唯一独立的配置空间(Configuration Space)与之对应。
和PCI总线一样,PCIe总线中的每一个功能(Function)都有一个唯一的标识符与之对应。这个标识符就是BDF(Bus,Device,Function)
ifconfig -a或者ip addr可以看到所有的网卡名
root@AIOserver-Pve:/# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp1s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a0:36:9f:26:47:0c brd ff:ff:ff:ff:ff:ff
3: enp1s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a0:36:9f:26:47:0d brd ff:ff:ff:ff:ff:ff
4: enp1s0f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a0:36:9f:26:47:0e brd ff:ff:ff:ff:ff:ff
5: enp1s0f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a0:36:9f:26:47:0f brd ff:ff:ff:ff:ff:ff
6: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
link/ether fc:34:97:0d:80:29 brd ff:ff:ff:ff:ff:ff
7: wlp5s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 44:af:28:a8:3b:d0 brd ff:ff:ff:ff:ff:ff
8: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether fc:34:97:0d:80:29 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.168/24 brd 192.168.1.255 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::fe34:97ff:fe0d:8029/64 scope link
valid_lft forever preferred_lft forever
而网卡名和BDF号的对应关系可以通过lshw命令找出
lshw -c network -businfo
随便挑一个网口,比如01:00.0 查看是否支持SR-IOV
lspci -s 01:00.1 -vvv | grep Capabilities
root@AIOserver-Pve:/# lspci -s 01:00.1 -vvv | grep Capabilities
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
Capabilities: [a0] Express (v2) Endpoint, MSI 00
Capabilities: [e0] Vital Product Data
Capabilities: [100 v2] Advanced Error Reporting
Capabilities: [140 v1] Device Serial Number a0-36-9f-ff-ff-26-47-0c
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
Capabilities: [1a0 v1] Transaction Processing Hints
Capabilities: [1d0 v1] Access Control Services
看到SR-IOV就说明支持,关于I350-T4这块卡,还有的扯,这里就不展开了
如果你和我一样买的是dell的OEM版拆机卡,通过各种方法判定都是非寨卡,也的确识别出是I350T4,但此处就是不显示SR-IOV。那么很有可能是固件太旧不支持SR-IOV,比如我的这块生产于2013年,固件版本14.多,但查询才发现15.0.1才开始支持SR-IOV,但这个版本在win10下打不开,换成19.0的就好了。
通过ethtool -i 网卡名查看固件版本
root@AIOserver-Pve:/# ethtool -i enp1s0f0
driver: igb
version: 5.6.0-k
firmware-version: 1.67, 0x80000fb7, 19.0.12
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
firmware-version行最后的数字就代表固件版本号,去dell官网找较新版本的固件升级包。这里推荐在win10下进行升级固件的操作,linux下太多问题最终没有成功,但我的目的只是升级固件,没必要死磕
https://www.dell.com/support/home/zh-cn/drivers/driversdetails?driverid=gk57c
其他OEM版本请自行去官网查找
成功之后就可以添加VF网口了,在此之前通过
cat /sys/bus/pci/devices/0000:BDF号/sriov_totalvfs 查看网口最大支持的VF网口数量
root@AIOserver-Pve:/# cat /sys/bus/pci/devices/0000:01:00.0/sriov_totalvfs
7
添加VF网口 echo 7 > /sys/class/net/网卡名/device/sriov_numvfs ,也可以vi打开文件根据自己的需要修改
正常情况下无任何回显
再次查看
root@AIOserver-Pve:/# echo 7 > /sys/class/net/enp1s0f0/device/sriov_numvfs
root@AIOserver-Pve:/# lspci|grep Eth
00:1f.6 Ethernet controller: Intel Corporation Device 0d55
01:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
02:10.0 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)
02:10.4 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)
02:11.0 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)
02:11.4 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)
02:12.0 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)
02:12.4 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)
02:13.0 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)
本文链接:https://www.kinber.cn/post/2889.html 转载需授权!
推荐本站淘宝优惠价购买喜欢的宝贝: