https://www.ibm.com/support/knowledgecenter/en/ssw_aix_61/com.ibm.aix.networkcomm/roce_rdmaconfiguration.htm

The PCIe2 10 GbE RoCE Adapter is preconfigured to operate in the AIX® RoCE mode. A network that uses RDMA provides better performance than an adapter that is used as a NIC for network-intensive applications. This mode is often helpful for network storage or high-performance computing.

AIX RoCE configuration requires the use of libraries or interfaces such as the following:

• The Direct Access Programming Library (uDAPL), which is used by the DB2® database system
• The Message Passing Interface (MPI), which is used by high performance computing (HPC)

Figure 1 displays the output when the adapter is running in the AIX RoCE mode.

The PCIe2 10 GbE RoCE Adapter shows only one adapter instance when it is in the AIX RoCE mode, but it can have up to two ports. Use the ibstat command to determine how many ports are configured by completing the following steps:

1. Determine whether the icm kernel extension is configured by entering the following command:

   # lsdev -C | grep icm

2. If the icm kernel is not configured, configure it by entering the following command:

   # mkdev -c management -s infiniband -t icm

3. Run the ibstat command by entering the following command:

   # ibstat roce0

While the PCIe2 10 GbE RoCE Adapter is initially configured to use the AIX RoCE mode, you might need to switch back from the AIX NIC + OFED RoCE configuration. To switch over from the AIX NIC + OFED RoCE configuration to the AIX RoCE configuration, complete the following steps:

1. Verify that the adapter is in the AIX NIC + OFED RoCE mode by entering the following command:

   # lsdev -C -c adapter

   The output of the lsdev command is similar to the example in Figure 1.
2. Stop the TCP/IP traffic and detach the IP interfaces by entering the following command:

   # ifconfig en1 down detach; ifconfig en2 down detach

3. Delete or put the NIC instances in a defined state by entering one of the following commands:
   • # rmdev -d -l ent1; rmdev -d -l ent2
   • # rmdev -l ent1; rmdev -l ent2
   The rmdev -l ent1; rmdev -l ent2 command retains the definition of the Ethernet devices so that you can use it the next time that you create instances.
4. Change the attribute of the hba stack_type from ofed (AIX NIC + OFED RoCE) to aix_ib (AIX RoCE) by entering the following command:

   # chdev -l hba0 -a stack_type=aix_ib 

5. Run the configuration manager tool so that the host bus adapter can configure the PCIe2 10 GbE RoCE Adapter as an AIX RoCE adapter by entering the following command:

   # cfgmgr

6. Verify that the adapter is now running in the AIX RoCE configuration by entering the following command:

   # lsdev -C -c adapter

   The following example shows the results when you run the lsdev command for adapters, and the adapter is configured in the AIX RoCE mode.
   Figure 1. Example output of the lsdev command for the adapters when it is using the AIX RoCE configuration

   roce0 Avaliable 00-00-00 PCIe2 10GbE RoCE Converged Network Adapter
   hba0 Available  00-00-00 PCIe2 10GbE RoCE Converged Host Bus Adapter (b315506714101604)

    

    

7. https://www.ibm.com/support/knowledgecenter/en/ssw_aix_61/com.ibm.aix.networkcomm/roce_rdmaconfiguration.htm