×

ProxmoxVE 6.4-13(PVE) 开启SR-IOV INTEL I350-T4 手把手教程

hqy hqy 发表于2022-11-10 09:48:47 浏览917 评论0

抢沙发发表评论

ProxmoxVE 6.4-13(PVE) 开启SR-IOV INTEL I350-T4 手把手教程



Single root input/output virtualization(SR-IOV), 是一种先进的虚拟化技术,以网卡为例,它可以在一个物理功能PF(一个网口)上开出多个虚拟功能(虚拟网卡)。


为了令虚拟机与外界的连接高效可靠,往往会采用直通网卡(PCIE Passthrough)的方法,但若是物理机中存在较多的虚拟机,这一方法便不再可行。此外,采取直通网卡的诸多虚拟机之间,并没有网络连接。


SR-IOV针对性的解决了以上两个问题,开启SR-IOV之后,物理网口PF与虚拟网口VFs(可以存在多个)间会建立一个虚拟交换机,这个虚拟交换机具有很好的性能,往往还支持RDMA等高级功能。倘若将VF直通进入虚拟机,这一虚拟机就会与外界、与其他挂在同一PF下的虚拟机,建立高效的连接。这些连接是绕过宿主机内核的,这将带来极好的网络性能。


首先要在主板中打开SR-IOV选项,华擎以及华硕的绝大部分主板都是支持的,b150这种也可能支持,华擎的有些bios内没有选项,但实际上是支持的


查看网是否正确识别出来,lspci -nn|grep Eth


root@AIOserver-Pve:/sys/bus/pci/devices/0000:01:00.0# lspci -nn|grep Eth

00:1f.6 Ethernet controller [0200]: Intel Corporation Device [8086:0d55]

01:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)

01:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)

01:00.2 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)

01:00.3 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)

最前面的数字是设备的BDF号


每一个PCIe设备可以只有一个功能(Function),即Fun0。也可以拥有最多8个功能,即多功能设备(Multi-Fun)。不管这个PCIe设备拥有多少个功能,其每一个功能都有一个唯一独立的配置空间(Configuration Space)与之对应。


和PCI总线一样,PCIe总线中的每一个功能(Function)都有一个唯一的标识符与之对应。这个标识符就是BDF(Bus,Device,Function)


ifconfig -a或者ip addr可以看到所有的网卡名 


root@AIOserver-Pve:/# ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: enp1s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000

    link/ether a0:36:9f:26:47:0c brd ff:ff:ff:ff:ff:ff

3: enp1s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000

    link/ether a0:36:9f:26:47:0d brd ff:ff:ff:ff:ff:ff

4: enp1s0f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000

    link/ether a0:36:9f:26:47:0e brd ff:ff:ff:ff:ff:ff

5: enp1s0f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000

    link/ether a0:36:9f:26:47:0f brd ff:ff:ff:ff:ff:ff

6: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000

    link/ether fc:34:97:0d:80:29 brd ff:ff:ff:ff:ff:ff

7: wlp5s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000

    link/ether 44:af:28:a8:3b:d0 brd ff:ff:ff:ff:ff:ff

8: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000

    link/ether fc:34:97:0d:80:29 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.168/24 brd 192.168.1.255 scope global vmbr0

       valid_lft forever preferred_lft forever

    inet6 fe80::fe34:97ff:fe0d:8029/64 scope link

       valid_lft forever preferred_lft forever

而网卡名和BDF号的对应关系可以通过lshw命令找出


lshw -c network -businfo

随便挑一个网口,比如01:00.0  查看是否支持SR-IOV


lspci -s 01:00.1 -vvv | grep Capabilities


root@AIOserver-Pve:/# lspci -s 01:00.1 -vvv | grep Capabilities

        Capabilities: [40] Power Management version 3

        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+

        Capabilities: [70] MSI-X: Enable+ Count=10 Masked-

        Capabilities: [a0] Express (v2) Endpoint, MSI 00

        Capabilities: [e0] Vital Product Data

        Capabilities: [100 v2] Advanced Error Reporting

        Capabilities: [140 v1] Device Serial Number a0-36-9f-ff-ff-26-47-0c

        Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)

        Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)

        Capabilities: [1a0 v1] Transaction Processing Hints

        Capabilities: [1d0 v1] Access Control Services

看到SR-IOV就说明支持,关于I350-T4这块卡,还有的扯,这里就不展开了


        如果你和我一样买的是dell的OEM版拆机卡,通过各种方法判定都是非寨卡,也的确识别出是I350T4,但此处就是不显示SR-IOV。那么很有可能是固件太旧不支持SR-IOV,比如我的这块生产于2013年,固件版本14.多,但查询才发现15.0.1才开始支持SR-IOV,但这个版本在win10下打不开,换成19.0的就好了。

0b01f137d313b3b115d3c01519103a90_watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RhbmdvYnJhdm8=,size_16,color_FFFFFF,t_70.png

通过ethtool -i 网卡名查看固件版本


root@AIOserver-Pve:/# ethtool -i enp1s0f0

driver: igb

version: 5.6.0-k

firmware-version: 1.67, 0x80000fb7, 19.0.12

expansion-rom-version:

bus-info: 0000:01:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

firmware-version行最后的数字就代表固件版本号,去dell官网找较新版本的固件升级包。这里推荐在win10下进行升级固件的操作,linux下太多问题最终没有成功,但我的目的只是升级固件,没必要死磕


https://www.dell.com/support/home/zh-cn/drivers/driversdetails?driverid=gk57c


其他OEM版本请自行去官网查找


成功之后就可以添加VF网口了,在此之前通过


cat /sys/bus/pci/devices/0000:BDF号/sriov_totalvfs 查看网口最大支持的VF网口数量


root@AIOserver-Pve:/# cat /sys/bus/pci/devices/0000:01:00.0/sriov_totalvfs

7

添加VF网口 echo 7 > /sys/class/net/网卡名/device/sriov_numvfs ,也可以vi打开文件根据自己的需要修改


正常情况下无任何回显


再次查看


root@AIOserver-Pve:/# echo 7 > /sys/class/net/enp1s0f0/device/sriov_numvfs

root@AIOserver-Pve:/# lspci|grep Eth

00:1f.6 Ethernet controller: Intel Corporation Device 0d55

01:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

01:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

01:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

01:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

02:10.0 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)

02:10.4 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)

02:11.0 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)

02:11.4 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)

02:12.0 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)

02:12.4 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)

02:13.0 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)


打赏

本文链接:https://www.kinber.cn/post/2889.html 转载需授权!

分享到:


推荐本站淘宝优惠价购买喜欢的宝贝:

image.png

 您阅读本篇文章共花了: 

群贤毕至

访客