Live Partition Mobility
Live Partition Mobility (LPM) allows you to migrate partitions from one physical server to another while partition is running.
When all requirements are satisfied and all preparation tasks are completed, then HMC verifies and validates the LPM environment. This HMC function has the responsibility of checking that all hardware and software prerequisites are met. If this validation turns out to be successful, then partition migration can be initiated by using the HMC GUI or the HMC command-line interface.
Types of LPM:
- Active partition migration is the ability to move a running LPAR to another system without disrupting the operation of the LPAR.
- Inactive partition migration allows to move a powered-off LPAR from one system to another.
- Suspended partition migration allows to move a suspended LPAR
PowerVM allows up to 8 concurrent migrations per Virtual I/O Server and up to 16 per system.
-------------------------------------------
Mover service partition (MSP):
MSP is an attribute of the Virtual I/O Server partition. (This has to be set on HMC for the VIOS LPAR). Two mover service partitions are involved in an active partition migration: one on the source system, the other on the destination system. Mover service partitions are not used for inactive migrations.
-------------------------------------------
Virtual asynchronous services interface (VASI):
The source and destination mover service partitions use this virtual device to communicate with the POWER Hypervisor to gain access to partition state. The VASI device is included on the Virtual I/O Server, but is only used when the server is declared as a mover service partition.
--------------------------------------------
LPM overview:
1. Partition profile (active profile only) copied from source to target FSP.
2. Storage is configured on the Target.
3. Mover service partitions (MSP) is activated.
4. Partition migration started, copying memory pages (retransfer necessary pages)
5. When majority of memory pages has moved, system activated on target.
6. Final memory pages moved.
7. Cleanup storage and network traffic.
8. Storage resources are deconfigured from the source.
9. Partition profile removed from source server
Active partition migration involves moving the state of a partition from one system to another while the partition is still running. During the migration the memory of the LPAR will be copied over to the destination system. Because the partition is active, a portion of the memory will be changed during the transfer. The hypervisor keeps track of these changed memory pages, on a dirty page list and retransfers them as necessary.
Live Partition Mobility does not make any changes to the network setup on the source and destination systems. It only checks that all virtual networks used by the mobile partition have a corresponding shared Ethernet adapter on the destination system.
The time necessry for an LPM depends on the LPAR memory size, the LPAR memory activity (writes) and the network bandwith between source and destination. (dedicated LPM network with at least 1Gbps is recommended). When running 8 concurrent migrations through a Virtual I/O Server it is recommended to use a 10 Gbps network. (If there are high speed network transfers that can generate extra CPUs on VIOS side, this could also slow down LPM). If multiple mover service partitions are available on either the source or destination systems, it is a good idea to distribute the load among them.
A single HMC can control several concurrent migrations. The maximum number of concurrent migrations is limited by the processing capacity of the HMC and contention for HMC locks. If more LPMs are done concurrently, and the number of migrations grows, the setup time using the GUI can become long. In this case CLI could be faster.
--------------------------------------------------
Requirements for LPM:
(Most of them are checked by validation process)
(At the end of PowerVM Introduction and Configuration Redbook, there is a good list with pictures, where to check each.)
Hardware:
- Power6 or later systems
- System should be managed by at lease one HMC or IVM (if dual HMC, both on same level)
- If different HMCs are used for source and dest., both HMC should on the same network (so they can communicate with each other)
- Migration readiness of source and dest. (for example a server running on battery power is not ready, validation will check this)
- The destination system must have enough processor and memory resources to host the mobile partition
VIOS:
- PowerVM Enterprise Edition with Virtual I/O Server (or dual VIOSes) (version 1.5.1.1 or higher)
- Working RMC connection between HMC and VIOS
- VIOS must be designated as a mover service partition on source and destination
- VIOS must have enough virtual slots on the destination server
- If virtual switch is used, it has to be the same name on source and destination side
- VIOS on both system must have SEA configured to bridge to the same Ethernet network used by the LPARs
- VIOS on both system must be capable of providing access to all disk resources to the mobile partition
- If VSCSI is used it must be accessible by both VIO Servers (on source and destination systems)
- If NPIV is used physical adapter max_xfer_size should be the same or greater at dest.side (lsattr -El fcs0 | grep max_xfer_size)
LPAR:
- AIX version must be AIX 6.1 or AIX 7.1
- Working RMC connection between HMC and LPAR
- LPAR has a unique name (cannot be migrated if LPAR name is already used on destination server)
- Migration readiness (LPAR in crashed or failed state cannot be migrated, maybe a reboot is needed, validation will checkt his)
- No physical adapters may be used by the mobile partition during the migration
- No logical host Ethernet adapters
- LPAR should have a virtual Ethernet adapter
- The LPAR we want to migrate cannot be a VIO Server
- The mobile partition’s network and disk access must be virtualized by using one or more Virtual I/O Servers
- All virtual networks of the LPAR (VLANs), must be availake on destination server
– The disks used by the mobile partition must be accessed through virtual SCSI, virtual Fibre Channel-based mapping, or both.
- If VSCSI is used no lv or files as backing devices (only LUNs can be mapped)
- If NPIV is used, each VFC client adapter must have a mapping to a VFC server adapter on VIOS
- If NPIV is used, at least one LUN should be mapped to the LPAR`s VFC adapter.
- LPAR is not designated as a redundant error path reporting partition
- LPAR is not part of an LPAR workload group (it can be dynamically removed from a group)
- LPAR is not using huge pages (for inactive migration it can use)
- LPAR is not using Barrier Synchronization Register (BSR) arrays (for inactive migration it can use)
--------------------------------------------------
Some additional notes:
- This is not a replacement for PowerHA solution or a Disaster Recovery Solution.
- The partition data is not encrypted when transferred between MSPs.
- Ensure that the logical memory block (LMB) size is the same on the source and destination systems.
(In ASMI or "lshwres -r mem -m <managed system> --level sys -F mem_region_size")
--------------------------------------------------
lslparmigr -r sys -m <system> shows how many concurrent migrations are possible (num_active_migr._supported)
lslparmigr -r lpar -m source_sys list status of lpars (lpar_id will be shown as well)
migrlpar -o v -t dest_sys -m source_sys --id 1 validation of lpar (id) for migration
echo $? if return code is 0, validation was successful
migrlpar -o m -t dest_sys -m source_sys -p lpar1 & migrating lpar
lssyscfg -r lpar -m source_sys -F name,state show state
lslparmigr -r lpar -m source_sys -F name,migration_state,bytes_transmitted,bytes_remaining
--------------------------------------------------
nmon -> p (it will show details useful for migration)
--------------------------------------------------
Steps needed for LPM via HMC GUI:
1. choose LPAR on HMC -> Operations -> Mobility
2. choose Validate (Destination system name will be filled automatically or choose one) other settings can be leaved as it is -> Validate
3. after validation, choose slot IDs what will be used on destination system (change other settings if needed)
4. Migrate -> it will show a progress bar and inform you if it is done
--------------------------------------------------
HSCLA27C The operation to get the physical device location for adapter ...
on the virtual I/O server partition ... has failed.
Solution for me was to remove unnecessary NPIV adapters and mappings.
(I had NPIV configured for that LPAR, but no disk was assigned to that LPAR. After removing this NPIV config, LPM was successful.)
--------------------------------------------------
Live Partition Mobility (LPM) allows you to migrate partitions from one physical server to another while partition is running.
When all requirements are satisfied and all preparation tasks are completed, then HMC verifies and validates the LPM environment. This HMC function has the responsibility of checking that all hardware and software prerequisites are met. If this validation turns out to be successful, then partition migration can be initiated by using the HMC GUI or the HMC command-line interface.
Types of LPM:
- Active partition migration is the ability to move a running LPAR to another system without disrupting the operation of the LPAR.
- Inactive partition migration allows to move a powered-off LPAR from one system to another.
- Suspended partition migration allows to move a suspended LPAR
PowerVM allows up to 8 concurrent migrations per Virtual I/O Server and up to 16 per system.
-------------------------------------------
Mover service partition (MSP):
MSP is an attribute of the Virtual I/O Server partition. (This has to be set on HMC for the VIOS LPAR). Two mover service partitions are involved in an active partition migration: one on the source system, the other on the destination system. Mover service partitions are not used for inactive migrations.
-------------------------------------------
Virtual asynchronous services interface (VASI):
The source and destination mover service partitions use this virtual device to communicate with the POWER Hypervisor to gain access to partition state. The VASI device is included on the Virtual I/O Server, but is only used when the server is declared as a mover service partition.
--------------------------------------------
LPM overview:
1. Partition profile (active profile only) copied from source to target FSP.
2. Storage is configured on the Target.
3. Mover service partitions (MSP) is activated.
4. Partition migration started, copying memory pages (retransfer necessary pages)
5. When majority of memory pages has moved, system activated on target.
6. Final memory pages moved.
7. Cleanup storage and network traffic.
8. Storage resources are deconfigured from the source.
9. Partition profile removed from source server
Active partition migration involves moving the state of a partition from one system to another while the partition is still running. During the migration the memory of the LPAR will be copied over to the destination system. Because the partition is active, a portion of the memory will be changed during the transfer. The hypervisor keeps track of these changed memory pages, on a dirty page list and retransfers them as necessary.
Live Partition Mobility does not make any changes to the network setup on the source and destination systems. It only checks that all virtual networks used by the mobile partition have a corresponding shared Ethernet adapter on the destination system.
The time necessry for an LPM depends on the LPAR memory size, the LPAR memory activity (writes) and the network bandwith between source and destination. (dedicated LPM network with at least 1Gbps is recommended). When running 8 concurrent migrations through a Virtual I/O Server it is recommended to use a 10 Gbps network. (If there are high speed network transfers that can generate extra CPUs on VIOS side, this could also slow down LPM). If multiple mover service partitions are available on either the source or destination systems, it is a good idea to distribute the load among them.
A single HMC can control several concurrent migrations. The maximum number of concurrent migrations is limited by the processing capacity of the HMC and contention for HMC locks. If more LPMs are done concurrently, and the number of migrations grows, the setup time using the GUI can become long. In this case CLI could be faster.
--------------------------------------------------
Requirements for LPM:
(Most of them are checked by validation process)
(At the end of PowerVM Introduction and Configuration Redbook, there is a good list with pictures, where to check each.)
Hardware:
- Power6 or later systems
- System should be managed by at lease one HMC or IVM (if dual HMC, both on same level)
- If different HMCs are used for source and dest., both HMC should on the same network (so they can communicate with each other)
- Migration readiness of source and dest. (for example a server running on battery power is not ready, validation will check this)
- The destination system must have enough processor and memory resources to host the mobile partition
VIOS:
- PowerVM Enterprise Edition with Virtual I/O Server (or dual VIOSes) (version 1.5.1.1 or higher)
- Working RMC connection between HMC and VIOS
- VIOS must be designated as a mover service partition on source and destination
- VIOS must have enough virtual slots on the destination server
- If virtual switch is used, it has to be the same name on source and destination side
- VIOS on both system must have SEA configured to bridge to the same Ethernet network used by the LPARs
- VIOS on both system must be capable of providing access to all disk resources to the mobile partition
- If VSCSI is used it must be accessible by both VIO Servers (on source and destination systems)
- If NPIV is used physical adapter max_xfer_size should be the same or greater at dest.side (lsattr -El fcs0 | grep max_xfer_size)
LPAR:
- AIX version must be AIX 6.1 or AIX 7.1
- Working RMC connection between HMC and LPAR
- LPAR has a unique name (cannot be migrated if LPAR name is already used on destination server)
- Migration readiness (LPAR in crashed or failed state cannot be migrated, maybe a reboot is needed, validation will checkt his)
- No physical adapters may be used by the mobile partition during the migration
- No logical host Ethernet adapters
- LPAR should have a virtual Ethernet adapter
- The LPAR we want to migrate cannot be a VIO Server
- The mobile partition’s network and disk access must be virtualized by using one or more Virtual I/O Servers
- All virtual networks of the LPAR (VLANs), must be availake on destination server
– The disks used by the mobile partition must be accessed through virtual SCSI, virtual Fibre Channel-based mapping, or both.
- If VSCSI is used no lv or files as backing devices (only LUNs can be mapped)
- If NPIV is used, each VFC client adapter must have a mapping to a VFC server adapter on VIOS
- If NPIV is used, at least one LUN should be mapped to the LPAR`s VFC adapter.
- LPAR is not designated as a redundant error path reporting partition
- LPAR is not part of an LPAR workload group (it can be dynamically removed from a group)
- LPAR is not using huge pages (for inactive migration it can use)
- LPAR is not using Barrier Synchronization Register (BSR) arrays (for inactive migration it can use)
--------------------------------------------------
Some additional notes:
- This is not a replacement for PowerHA solution or a Disaster Recovery Solution.
- The partition data is not encrypted when transferred between MSPs.
- Ensure that the logical memory block (LMB) size is the same on the source and destination systems.
(In ASMI or "lshwres -r mem -m <managed system> --level sys -F mem_region_size")
lslparmigr -r sys -m <system> shows how many concurrent migrations are possible (num_active_migr._supported)
lslparmigr -r lpar -m source_sys list status of lpars (lpar_id will be shown as well)
migrlpar -o v -t dest_sys -m source_sys --id 1 validation of lpar (id) for migration
echo $? if return code is 0, validation was successful
migrlpar -o m -t dest_sys -m source_sys -p lpar1 & migrating lpar
lssyscfg -r lpar -m source_sys -F name,state show state
lslparmigr -r lpar -m source_sys -F name,migration_state,bytes_transmitted,bytes_remaining
nmon -> p (it will show details useful for migration)
--------------------------------------------------
Steps needed for LPM via HMC GUI:
1. choose LPAR on HMC -> Operations -> Mobility
2. choose Validate (Destination system name will be filled automatically or choose one) other settings can be leaved as it is -> Validate
3. after validation, choose slot IDs what will be used on destination system (change other settings if needed)
4. Migrate -> it will show a progress bar and inform you if it is done
--------------------------------------------------
HSCLA27C The operation to get the physical device location for adapter ...
on the virtual I/O server partition ... has failed.
Solution for me was to remove unnecessary NPIV adapters and mappings.
(I had NPIV configured for that LPAR, but no disk was assigned to that LPAR. After removing this NPIV config, LPM was successful.)
--------------------------------------------------