SR-IOV - vNIC

 

SR-IOV - vNIC

SR-IOV

SR-IOV (Single Root IO Virtualization) is a network virtualization technology. The basic idea is, that a physical device can have multiple virtual instances of itself, and these can be assigned to any LPARs running on the managed system. We have only 1 physical adapter, but each LPAR will think it has its own dedicated adapter. This is achieved by creating logical ports above the physical ports on the adapter. These ports will be exist in the Hypervisor (Firmware), so no VIOS is necessary. In order to see new SR-IOV menu points in HMC GUI, Firmware and HMC should be in a correct level.

Dedicated Mode - Shared Mode:
An SR-IOV capable adapter is either in dedicated mode or shared mode.
In dedicated mode, the adapter is owned by one LPAR. Physical ports of the adapter are owned by that partition. (Usual old config.)
In shared mode, adapter is owned by the Hypervisor. In this mode logical ports can be created and these can be assigned to any LPAR.

Logical Port (LP) - Virtual Function (VF)
A Virtual Function (VF) is a general term used by PCI standards.  On IBM Power Systems, SR-IOV implements VFs as logical ports. Each logical port is associated with a physical port of the adapter. (Basically LPs and VFs are the same. In general documentations the term VF is used many times, but during SR-IOV configuration on HMC, I met only LP, so I`ll use LP here. In these standards another term, the Physical Function (PF) also exists, which is the Physical Port.)

------------------------------------

Capacity (SR-IOV desired bandwidth)

During Logical Port (Virtual Function) creation the desired capacity need to be configured. This is similar to the Entitled Capacity (CPU) setting, just here % is used. The configured value is the desired minimum bandwidth in percentage. It is not capped, which means, if there is additional bandwidth that is not being used currently, it will be shared equally among all logical ports. Assignments are made in increments of 2 %  and total assignments for a single port can not exceed 100%. Capacity cannot be changed dynamically. (It is possible to change in profile and after profile activation is needed.)

So, if an LPAR needs, it will have % desired outgoing bandwidth. If additional outgoing bandwidth is available, any partition can use it. If a partition doesn’t need its minimum, that bandwidth is available to other partitions until the owning partition needs it. Capacity settings don’t have any influence on the incoming bandwidth.

------------------------------------

SR-IOV and LPM:

From a virtualization perspective, SR-VIO logical ports are seen as physical adapters at OS level, therefore operations like Live Partition Mobility are not supported when an SR-IOV logical port is configured on the partition (LPAR B).

If a Logical Port is part of a SEA (which bridge traffic from client partitions to the physical network) then client LPAR has only a Virtual Ethernet Adapter (LPAR A), so it can continue using Live Partition Mobility.

------------------------------------

SR-IOV and Link Aggregation

In an LACP configuration multiple primary SR-IOV logical ports are allowed. When LACP (IEEE802.3ad,) configured with multiple main logical ports, only SR-IOV logical ports can be part of the link aggregation and only 1 single logical port can be configured per physical port.

So, with LACP only one logical port per physical port can be used.
(The second configuration, with more than one logical port assigned to a physical port, will not work.)

To prevent users from adding a logical port to the physical port when LACP being used, you can set the logical port capacity to 100%.

------------------------------------

SR-IOV and Etherchannel (NIB):

In an active-passive configuration (Network Interface Backup), SR-IOV logical port can be primary (active) or backup (passive), or both. If more than one primary adapter is configured in an Etherchannel, then SR-IOV logical port cannot be a primary adapter. When an SR-IOV logical port is configured in an active-passive configuration, it must be configured to detect when to fail over from primary to the backup adapter. This can be achieved by configuring an IP address to ping.

------------------------------------

SR-IOV Configuration:
(Here HMC classical GUI has been used, not enhanced HMC GUI.)


1. verify preprequisites: HMC level, Firmware level, compatible SR-IOV adapter, Man. Sys. capabilities (SR-IOV capable: True)

2. change adapter from Dedicated to Shared mode: Man. Sys. Properties --> IO tab --> choose adapter --> click on shared mode

(This happens at Man. Sys. level, so adapter should not be assigned to any LPARs. If you check again after modification, you will see adapter will be owned by Hypervisor. It is possible to switch back to dedicated mode, just already configured logical ports must be de-configured prior to that.)

3. Physical port config: Man. System --> Properties --> IO --> choose an SR-IOV adapter --> choose a port
Here you can configure speed (10Gb), MTU size, label...

4.Logical Port config: Logical Port is mapped to an LPAR so an LPAR has to be chosen before config. It can be done as a DLPAR operation, or it can be done in LPAR profile.

Add a logical port to a Physical Port:
DLPAR: choose LPAR --> Dynamic part. --> SR-IOV Log. Ports --> Action --> Add Log. Port --> choose a phys. port to conf. additional Log. Port.
PROFILE: In LPAR profile, SR-IOV Log. Ports -->  SR-IOV Menu --> Add Log. Port -->  choose a physical port to configure additional Log. Port

After new window will pop up:

Capacity:     Set desired percentage of Log. Port
Promiscuous:  if the logical port will be assigned to a VIOS in a SEA, then enabling promiscuous mode is required.
              (Promiscuous mode can be enabled on only one logical port per physical port.)
              When promiscuous mode is enabled, Allow All VLAN IDs and  Allow all O/S Defined MAC Addresses are the only options available.

After finishing it, on OS level a new adapter will be configured with VF in the name (cfgmgr may needed)

# lsdev -Cc adapter
ent0    Available 02-00 4-Port Gigabit Ethernet PCI-Express Adapter (e414571614102004)
ent1    Available 02-01 4-Port Gigabit Ethernet PCI-Express Adapter (e414571614102004)
ent2    Available 02-02 4-Port Gigabit Ethernet PCI-Express Adapter (e414571614102004)
ent3    Available 02-03 4-Port Gigabit Ethernet PCI-Express Adapter (e414571614102004)
ent4    Available 0C-00 PCIe3 4-Port 10GbE SR Adapter VF(df1028e21410e304)

# entstat -d ent4
...
...
VF Minimum Bandwidth: 24%
VF Maximum Bandwidth: 100%

------------------------------------
------------------------------------
------------------------------------

vNIC

vNIC (Virtual Network Interface Controller) is a new type of network adapter on AIX 7.1 TL4 (or later) or on AIX 7.2. vNIC adapters are based on SR-IOV technology and with those LPM is possible.

# lsdev -Cc adapter
ent0   Available   Virtual I/O Ethernet Adapter (l-lan)
ent1   Available   Virtual I/O Ethernet Adapter (l-lan)
ent2   Available   Virtual I/O Ethernet Adapter (l-lan)
ent3   Available   Virtual NIC Client Adapter (vnic)
ent4   Available   Virtual NIC Client Adapter (vnic)