# SRIOV troubleshooting with Linux

SRIOV boogaloo

### Basics of SRIOV

SRIOV is a specific method to bind a virtual NIC to the physical NIC of the hypervisor directly. There is no bridge/veth pair or faking in place. The VM is talking directly with the metal.

### Concepts and terminology

• The PF is the “Physical Interface” - the NIC card itself.
• The VF is the “Virtual Function” - the fake NIC that is bound to the VM.
• You can create multiple VF that are all bound to the same PF.
• Each of those VF can be bound to the same VM or multiple different VMs.
• Card drivers/manufacturers have different limits on the number of VF per PF. There is a performance hit after a certain number - I’ve seen it work on X710 cards, X520 (82599) - both from Intel. Mellanox also seems to support it. There have a lot of documentation regarding all the different settings that can be tweaked.

### Why?

• It mostly comes down to a performance requirements.
• It’s “simpler” to manage as there is nothing between you and the hardware.
• Very common in NFV deployments where low latency, high throughput is required.
• Extremely dependant on drivers, manufacturer. YMMV depending on the mix of both.
• You can bond VF on two different SRIOV ports to allow HA over two different TORs.

### Things you won’t be able to do.

• The kernel loses all control over the VF.
• You cannot tcpdump a VF.
• You can tcpdump a PF (the physical port itself) but results will vary. With the X710 cards, you will only see inbound (TOR) traffic (maybe even only broadcast packets).

### Settings to tweak

• Trusted VF
• Allow to VF to enter promiscuous mode. Allow the VF to run a packet capture. There is some security concern as when the VF is trusted, the VM with the VF can see the IN/OUT traffic accross all other VFs (who might not be owned by the same tenant). Does not matter too much if the compute is only use by a single tenant.
• ip link show \$SRIOV_PORT_NAME