root@ubuntu:/etc/apt#
root@ubuntu:/etc/apt# cat sources.list
#
deb [trusted=yes] file:/cdrom/dists/stable/main/binary-ppc64el ./
root@ubuntu:/etc/apt#
root@ubuntu:/etc/apt# cat sources.list
#
deb [trusted=yes] file:/cdrom/dists/stable/main/binary-ppc64el ./
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/html/release_notes/sect-subscriptions
http://www.cbs1.com.my/WebLITE/Applications/news/uploaded/docs/Linux%20on%20Power%20Update%20and%20Strategy%20V1.0.pdf
https://www.ibm.com/developerworks/community/blogs/mhhaque/entry/How_Install_RHVM_Red_Hat_oVirt_Virtualization_Manager_on_Power_System_VM_or_LPAR?lang=en
https://www.ibm.com/developerworks/library/l-rhv-environment-trs/
Hot add, remove, rescan of SCSI devices on Linux (0) | 2018.07.09 |
---|---|
Redhat Nouveau 내장그래픽카드 nvidia 드라이버 설치 삭제 (0) | 2018.07.03 |
CentOS 7 bonding nmcli (0) | 2017.11.01 |
gnome login screen id (0) | 2016.08.11 |
Linux semaphore (0) | 2016.07.11 |
https://www.ibm.com/developerworks/linux/library/l-openpower-firmware-ipmi/index.html
This article describes the capabilities and features of OpenPOWER hardware and reliability, availability, and serviceability (RAS) components. This article also describes different OpenPOWER firmware features that use Intelligent Platform Management Interface (IPMI).
Firmware plays a critical role in the initialization and booting processes in servers. IBM Power Systems servers are developed based on the IBM® POWER® processor architecture and can be categorized into the following types:
Based on the management controller, there are two types of Power non-virtualization systems:
This article focuses on OpenPOWER BMC-based Power non-virtualization systems that are running in the OPAL mode. The firmware for the OpenPOWER systems is a combination of the BMC firmware component and the host firmware component.
The host firmware is the main boot firmware that stores in the processor NOR (PNOR) flash chip, and is used to initialize the host processor and make the system boot until runtime state after which the boot process is taken over by the Linux® operating system. OpenPOWER systems firmware has the following boot sequence after you power on the system:
The important OpenPOWER firmware IPMI features that are used to monitor, control, and debug the OpenPOWER system are described here.
For all the IPMI features described in this article, invoke the following out-of-band ipmitool
command as the base string when you run the commands for all operations:
$IPMI_CMD = ipmitool -I lanplus -H $BMCIP -U $USER -P $PASSWORD $BMCIP BMC IP address $USER BMC IPMI Username $PASSWORD BMC IPMI Password $IPMI_CMD IPMI tool command invocation $CMD = $IPMI_CMD $Operation |
All system power control operations are controlled by the BMC. The different power control operations are power status, power on, power off, power cycle, and power reset.
$IPMI_CMD chassis power status
$IPMI_CMD chassis power on
To track the power status during the boot process of an OpenPOWER system, the host status sensor provides a system Advanced Configuration and Power Interface (ACPI) power state. This sensor is set to the following states during booting.
More details of sensors are provided in the Sensors section.
$IPMI_CMD chassis power soft
Before BMC issues hardware power off, it must set the host status sensor to the soft off state.
$IPMI_CMD chassis power off
On forced power off, the system immediately goes to the standby state irrespective of the operating system status, and BMC sets the host status sensor to the softoff state.
$IPMI_CMD chassis power reset
$IPMI_CMD chassis power cycle
$IPMI_CMD chassis power policy list
To set to a particular power policy, run the following command:
$IPMI_CMD chassis power policy <always-on/always-off/previous>
Sensors are software or hardware physical entities that are used to monitor the health of different system hardware devices or software components. There are two types of sensors.
$IPMI_CMD sensor list
or
$IPMI_CMD sdr elist
The following example shows the virtual sensor Host Status usage.
During system run-time, the sensor status shows as S0/G0: WORKING
, and when the system is in standby state or powered off state, the sensor status shows as S5/G2: SOFT-OFF
.
HOST STATUS | 04H | OK | 35.0 | S0/G0: WORKING HOST STATUS | 04H | OK | 35.0 | S5/G2: SOFT-OFF |
CPU Temp | 64h | ok | 3.0 | 42 degrees C Membuf Temp 0 | 65h | ok | 7.0 | 42 degrees C DIMM Temp 0 | 69h | ok | 32.0 | 28 degrees C DIMM Temp 1 | 6Ah | ns | 32.1 | No Reading |
This feature is mainly used to monitor and control system power and thermal levels. BMC keeps monitoring certain temperatures (processor, DIMM, and so on) with the help of the OCC and sets the fan speed based on the current readings. You can use the power reading and capping functions to read and maintain the power level of the system. You can run the following commands to deal with platform power limits:
$IPMI_CMD dcmi power power < command > reading Get power related readings from the system get_limit Get the configured power limits set_limit Set a power limit option activate Activate the set power limit deactivate Deactivate the set power limit |
$IPMI_CMD dcmi power reading Instantaneous power reading: 237 Watts Minimum during sampling period: 233 Watts Maximum during sampling period: 240 Watts Average power reading over sample period: 236 Watts IPMI timestamp: Mon Nov 7 13:20:22 2016 Sampling period: 00000010 Seconds. Power reading state is: activated |
$IPMI_CMD dcmi power get_limit Current Limit State: Power Limit Active Exception actions: Log Event to SEL Power Limit: 1000 Watts Correction time: 1000 milliseconds Sampling period: 10 seconds |
$IPMI_CMD dcmi power set_limit limit 1050 Current Limit State: Power Limit Active Exception actions: Log Event to SEL Power Limit: 1050 Watts Correction time: 1000 milliseconds Sampling period: 10 seconds |
$IPMI_CMD dcmi power activate Power limit successfully activated |
$IPMI_CMD dcmi power deactivate Power limit successfully deactivated |
The BMC is the master of system event log repository and maintains all firmware system event logs. All low-level events in the system firmware happen during system boot and runtime. Also, all the critical failures in the hardware and firmware are logged in the BMC System event log repository. 64 KB of system event log and extended system event log data can be stored.
$IPMI_CMD SEL LIST 1 | 11/07/2016 | 01:44:49 | TEMPERATURE #0X30 | UPPER CRITICAL GOING HIGH | ASSERTED 2 | 11/07/2016 | 01:45:08 | VOLTAGE #0X60 | LOWER CRITICAL GOING LOW | ASSERTED 3 | 11/07/2016 | 01:45:25 | MEMORY #0X53 | CORRECTABLE ECC | ASSERTED |
System boot progress can be tracked by seeing the SOL console where hostboot and OPAL emit their appropriate progress codes or by updating system firmware progress and operating system boot sensors to their respective boot states (that is, processor initialization, motherboard initialization, and PCI initialization).
Hostboot progress codes:
3.42145|Ignoring boot flags, incorrect version 0x0 3.68858|ISTEP 6. 3 4.13466|ISTEP 6. 4 4.13535|ISTEP 6. 5 15.27375|HWAS|PRESENT> DIMM[03]=AAAAAAAAAAAAAAAA 15.27375|HWAS|PRESENT> Membuf[04]=CCCC000000000000 15.27376|HWAS|PRESENT> Proc[05]=C000000000000000 28.57643|ISTEP 6. 6 28.66284|ISTEP 6. 7 28.66337|ISTEP 6. 8 28.69192|ISTEP 6. 9 31.32017|ISTEP 6.10 31.36741|ISTEP 6.11 33.17372|ISTEP 6.12 33.17531|ISTEP 6.13 33.17579|ISTEP 7. 1 |
OPAL or Skiboot progress codes:
[ 44.095518490,5] OPAL skiboot-5.6.0-158-ga1e0a047b2a0 starting... [ 44.095526941,7] initial console log level: memory 7, driver 5 [ 44.095530349,6] CPU: P8 generation processor (max 8 threads/core) [ 44.095533485,7] CPU: Boot CPU PIR is 0x0068 PVR is 0x004d0200 [ 44.095536954,7] CPU: Initial max PIR set to 0x1fff [ 44.095993908,7] OPAL table: 0x300dc240 .. 0x300dc730, branch table: 0x30002000 [ 44.095999625,7] Assigning physical memory map table for unused [ 44.096003854,7] FDT: Parsing fdt @0xff00000 [ 44.099952132,6] CHIP: Initialised chip 0 from xscom@3fc0000000000 [ 44.100047333,5] CHIP: Chip ID 0000 type: P8 DD2.0 [ 44.100050694,7] XSCOM: Base address: 0x3fc0000000000 [ 44.100059469,7] XSTOP: XSCOM addr = 0x2010c82, FIR bit = 31 [ 44.100063231,6] MFSI 0:0: Initialized [ 44.100065694,6] MFSI 0:2: Initialized [ 44.100068188,6] MFSI 0:1: Initialized [ 44.100451213,5] LPC: LPC[000]: Initialized, access via XSCOM @0xb0020 [ 44.100459497,7] LPC: Default bus on chip 0x0 [ 44.100585382,6] MEM: parsing reserved memory from node /ibm,hostboot/reserved-memory [ 44.100602925,7] HOMER: Init chip 0 |
System VPD is stored on the BMC in the IPMI FRU inventory. The BMC collects the FRU data of the hardware that is directly connected to the BMC [for example, backplane, power supply, and voltage regulator module (VRM)]. Hostboot updates the processor, centaur, and DIMM VPD. OPAL updates Peripheral Component Interconnect Express (PCIe) VPD. By running the following command, you can collect the complete system VPD including data that contains manufacturing information, product name, serial, and part numbers.$IPMI_CMD fru print
The example FRU description for the processor is as follows:
FRU Device Description : CPU (ID 1) Board Mfg Date : Mon Jan 1 05:30:00 1996 Board Mfg : IBM Board Product : PROCESSOR MODULE Board Serial : YA1932735603 Board Part Number : 00UM003 Board Extra : ECID:019A007301180718050A000000C031C2 Board Extra : EC:20 |
SOL is a mechanism that redirects the input and output of a serial port of the remote system over LAN IP. BMC provides the IPMI SOL console with the help of Ethernet and serial ports that are attached to it. You can see the boot progress and failure messages on the console screen. SOL console is the main interface between the overall system and the user. By running the following command, you can connect to the SOL console: $IPMI_CMD sol activate
Also, BMC has 32 KB of circular log buffer that is assigned for SOL console data, which you can get from the BMC busy box.
# cat /extlog/sollog/ /extlog/sollog/SOLHostCapture.log /extlog/sollog/SOLHostCapture.log.1 /extlog/sollog/archive/ # cat /extlog/sollog/SOLHostCapture.log.1 17.88935|ISTEP 11.12 17.89009|ISTEP 11.13 17.89083|ISTEP 12. 1 17.98714|ISTEP 12. 2 18.08120|ISTEP 12. 3 18.11037|ISTEP 12. 4 18.47909|ISTEP 12. 5 18.48009|ISTEP 13. 1 18.56525|ISTEP 13. 2 18.64082|ISTEP 13. 3 18.64194|ISTEP 13. 4 18.67988|ISTEP 13. 5 18.68156|ISTEP 13. 6 19.57066|ISTEP 13. 7 19.74316|ISTEP 13. 8 19.89404|ISTEP 13. 9 |
When the system is at a checkstop (for example, when the processor is not able to complete any instructions for some time or it is in an impossible state) the corresponding console log data is archived for further debugging and analysis.
System time is maintained in the Real-Time Clock (RTC), which is controlled by the BMC. When the BMC boots the system, it sets its own time by reading the current Inter-Integrated Circuit (I2C) RTC time. The host (hostboot or OPAL) or the user can read or write the RTC time by running the following IPMI commands. BMC sets or obtains the current RTC time.
$IPMI_CMD sel time get 03/12/2017 09:32:12 $IPMI_CMD sel time set “03/12/2017 09:34:12” 03/12/2017 09:34:12 |
OpenPOWER systems have two sides of the PNOR firmware: Primary side and golden side. BMC always boots the system from the primary side of the PNOR firmware. If the primary side boot fails twice due to system checkstop, watchdog conditions (conditions where the watchdog timer detects malfunctions of the operating system or server and recovers from them), or any other reason, BMC boots the system from the golden side of the PNOR firmware. For this process, BMC uses a boot count sensor, which is initially set to two. During the start of every boot operation, BMC decrements this sensor value by one. When the system reaches to the user-accessible level (that is, petitboot or when Linux boots), OPAL resets this sensor value back to two. Whenever the value of the boot count reaches zero due to boot failure, BMC starts booting the system from the golden side of the PNOR firmware.
During any boot failures, run the following command to know the value of boot count:
$IPMI_CMD sensor list | grep -i "boot count" Boot Count | 0x0 | discrete | 0x0280| na | na | na | na | na | na |
The BIOS golden side sensor has two discrete values (0x0080, 0x0180). The value of this sensor determines the side of the PNOR firmware boot.
The following example shows that the value of this sensor is set to 0x0180, which means that the system boots from the golden side of the PNOR firmware:
$IPMI_CMD sensor list | grep -i golden BIOS Golden Side | 0x0 | discrete | 0x0180| na | na | na | na | na | na |
The following example shows that the value of this sensor is set to 0x0080, which means that the system boots from the primary side of the PNOR firmware:
$IPMI_CMD sensor list | grep -i golden BIOS Golden Side | 0x0 | discrete | 0x0080 | na | na | na | na | na | na |
Usually, systems have general availability firmware installed. If the firmware gets corrupted or you want to upgrade the firmware to install bug fixes, you can select the out-of-band firmware update method. System firmware is a combination of the host firmware component (PNOR) and the BMC firmware component. It is a Hardware Platform Management (.hpm) file. To update the firmware code (xxxx.hpm) by using the out-of-band method, you need to complete the following steps:
$IPMI_CMD chassis power off
$IPMI_CMD mc reset cold
Then wait until the BMC starts.
$IPMI_CMD raw 0x32 0xba 0x18 0x00
$IPMI_CMD hpm upgrade <xxxx.hpm file> -z 30000 force
Wait until the firmware upgrade is successful.
$IPMI_CMD chassis power on
Generally, most of the users want to see boot messages or any boot failures during system boot. To view early boot messages in a system boot, the VUART-to-VGA rendering feature is implemented in OpenPOWER systems. By using this feature, when you power the system on, you can see all the early boot messages in a local VGA console. The BMC renders the output of the host UART to the VGA display.
The boot loader (petitboot) has the following settings in the user interface:
All these features can also be controlled by the BMC. By using IPMI commands, you can enable or disable the turbo mode, select a boot device, and configure the host network.
The settings that are changed by using the IPMI commands take preference, and the settings that are changed at petitboot in nonvolatile random access memory (NVRAM) configuration are overridden. Also, system information provides complete firmware versions (BMC and PNOR versions, including the golden side version), and the BMC MAC address.
When a system is deployed to a customer, unauthenticated in-band IPMI must not be able to access certain BMC configuration information and functions. The OpenPOWER original equipment manufacturer (OEM) IPMI Lock
command can be run in an authenticated interface to lock certain IPMI commands. After BMC is in the IPMI lockdown mode, further unauthenticated in-band IPMI messages must satisfy the allowed commands, added in a list called white list that is matched by NetFn
command specification. To revert the BMC to a fully permitted configuration, you can run the OEM IPMI Unlock
command in an authenticated interface.
Lock IPMI interface
command: You can run this command to lock the IPMI interface for safer executions. Only a predefined set of commands are allowed in this mode of operations. This is similar to the safe mode operations. $IPMI_CMD raw 0x32 0xf3 0x4c 0x4f 0x43 0x4b 0x00; echo $? 0 # ipmitool chassis status Error sending Chassis Status command: Insufficient privilege level Get SEL time # ipmitool raw 0x0a 0x48; echo $? 00 ae c0 57 0 |
From these examples, you can see that the Chassis Status
command does not produce any output because the command is not listed in the white list. However, when you run the Get SEL Time
command, output is displayed because the command is available in the white list. Commands that are listed in the white list can work when BMC is in the IPMI lockdown mode.
Unlock IPMI Interface
command: By unlocking the IPMI interface, you can run all the commands again. $IPMI_CMD raw 0x32 0xF4 0x55 0x4e 0x4c 0x4f 0x43 0x4b 0x00 |
You can revert the modified system settings and modified configurations back to default settings. The OEM IPMI PNOR Reprovision
command in OpenPOWER resets the system to the default settings. BMC clears any persistent data that is set by the user. Currently, in PNOR, the erasable partitions are GARD (hardware guard entries), NVRAM (boot loader configuration), hostboot attribute overrides (turbo mode non-default value to default), and FIRDATA.
The following procedure shows the PNOR re-provision process for an NVRAM partition:
# nvram --print-config "common" Partition --------------------- # nvram --update-config test-name=test-value # nvram --print-config "common" Partition --------------------- test-name=test-value |
# $IPMI_CMD raw 0x3A 0x1C; echo $? [392812400894,5] IPMI: PNOR access requested |
00
indicates successful reprovision. 03
indicates that the re-provisioning is still in progress. # $IPMI_CMD raw 0x3A 0x1D; echo $? 03 0 # $IPMI_CMD raw 0x3A 0x1D; echo $? 03 0 [163479624724,5] IPMI: PNOR access released # $IPMI_CMD raw 0x3A 0x1D; echo $? 00 0 |
# nvram --print-config "common" Partition --------------------- test-name=test-value # reboot # nvram --print-config |
NVRAM data is erased after a PNOR reset or reprovision operation.
The IPMI raw command data might change between different OpenPOWER BMC vendors. However, the functionality remains the same. All the examples mentioned in this article mainly refer to American Megatrends BMC vendor-based OpenPOWER systems.
H/W check and status check (0) | 2018.07.23 |
---|---|
Amber off (0) | 2018.07.05 |
Scale-out LC System Event Log Collection Tool (0) | 2018.06.19 |
RHEL7: How to get started with CPU governor (0) | 2018.03.20 |
리눅스에서 LVM 구성 및 사용방법 (0) | 2018.03.06 |