https://www.certdepot.net/rhel7-get-started-cpu-governor/

 

RHEL7: How to get started with CPU governor.

 

 

 

 

Presentation of CPU Governor

The modern x86 processors provide two mechanisms to reduce power consumption when idle. Useful for a portable PC, it can also help for physical servers with limited workload.

However, don’t think about virtual machines, they are not concerned.

CPU Idle States

In the x86 architecture several CPU states, called C-states, have been defined, allowing systems to save power by decreasing CPU functionalities. These C-states are broadly similar across processors but the exacts details may vary.

  • C0: The processor is working as usual.
  • C1: The processor doesn’t execute any instruction but can start working again without any delay.
  • C2: The processor is stopped but keeps the complete state of its registers and caches. A delay is necessary for the processor to start working again.
  • C3: The processor is sleeping and doesn’t keep its caches. A longer delay than in C2 state is needed for the processor to start working again.

CPU Freq

CPU Freq or CPU speed scalling is a way to reduce power consumption by adjusting the clock speed of the processor.
Five CPU Freq governors are available in RHEL 7:

  • performance: with this static governor you get the highest possible clock frequency but without any power saving benefit. It is best suited for heavy workload when the CPU is almost never idle.
  • powersave: here your CPU gets the lowest possible clock frequency but at the cost of the lowest CPU performance. One serious drawback of this static governor happens when the system experiences unexpected high loads: in this case it can consume more power than other governors with higher clock frequencies. For this reason, it is not recommended to use it except when overheating is a problem.
  • ondemand: this dynamic governor sets the maximum clock frequency when system load is high and minimum clock frequency when idle, but without intermediate state. This allows the system to adjust power consumption according to system load but at the expense of latency. In case switches between idle and heavy workloads happen too often, performance and power saving can suffer. Otherwise, it is one of the best options.
  • userspace: with this governor, any process running as root can set the clock frequency.
  • conservative: this dynamic governor is very similar to the ondemand governor except it adjusts clock frequency more gradually. Instead of choosing between maximum and minimum clock frequencies, it selects the clock frequency according to usage. This more granular approach provides significant power saving but at the expense of an ever greater latency than the ondemand governor.

Installation Procedure

Install the kernel-tools package to get access to the cpupower command:

# yum install -y kernel-tools

Note: The kernel-tools package and the cpupower command are not strictly needed to manipulate CPU governors. They only provide a convenient interface. All operations can be done using the /sys/devices/system/cpu/ path and the echo command.

Basic Operations

To get the list of the various idle states supported by the CPU number 0 (available idle states are given for various types of servers), type:

# cpupower idle-info
CPUidle driver: intel_idle
CPUidle governor: menu
analyzing CPU 0:
...
Available idle states: POLL C1E-ATM C2-ATM C4-ATM C6-ATM (Atom CPU N2800)
Available idle states: POLL C1-IVB C1E-IVB C3-IVB C6-IVB (Xeon CPU E3-1245 V2)
Available idle states: POLL C1-NHM C1E-NHM C3-NHM C6-NHM (Core i5 CPU M 430)
Available idle states: POLL C1-HSW C1E-HSW C3-HSW C6-HSW C7s-HSW C8-HSW C9-HSW C10-HSW (Celeron 2961Y)
...

To get the same information for all the CPUs on a server, type:

# cpupower -c all idle-info

Note: The -c option can be replaced with –cpu.

To get the list of the available governors for the CPU number 0, type:

# cpupower frequency-info -g
analyzing CPU 0:
  available cpufreq governors: conservative userspace powersave ondemand performance

Note: The -g option can be replaced with –governors.

To get all the details about the available CPU frequencies for the CPU number 0, type:

# cpupower frequency-info
analyzing CPU 0:
 driver: acpi-cpufreq
 CPUs which run at the same hardware frequency: 0
 CPUs which need to have their frequency coordinated by software: 0
 maximum transition latency: 10.0 us
 hardware limits: 1.20 GHz - 2.27 GHz
 available frequency steps: 2.27 GHz, 2.27 GHz, 2.13 GHz, 2.00 GHz, 1.87 GHz, 1.73 GHz, 1.60 GHz, 1.47 GHz, 1.33 GHz, 1.20 GHz
 available cpufreq governors: conservative userspace powersave ondemand performance
 current policy: frequency should be within 1.20 GHz and 2.27 GHz.
                 The governor "conservative" may decide which speed to use
                 within this range.
 current CPU frequency: 1.20 GHz (asserted by call to hardware)
 boost state support:
 Supported: yes
 Active: yes
 1900 MHz max turbo 2 active cores
 1900 MHz max turbo 1 active cores

Note1: Without the -c option, only the information about the CPU number 0 is displayed.
Note2: The governor conservative is the current configuration.

To change the governor to performance for all the CPUs, type:

cpupower frequency-set -g performance
Setting cpu: 0
Setting cpu: 1
Setting cpu: 2
Setting cpu: 3

Note1: Without the -c option, all the CPUs are affected.
Note2: To only change for the CPUs number 0, 1 and 2, type: # cpupower -c 0-2 frequency-set -g performance
Note3: The -g option can be replaced with –governor.

# cpupower -c all frequency-info
...
 current policy: frequency should be within 1.20 GHz and 2.27 GHz.
                 The governor "performance" may decide which speed to use
                 within this range.
...

To specify the governor userspace for the CPU number 1 with a CPU frequency of 1.2GHz, type:

# cpupower -c 1 frequency-set -f 1.2
Setting cpu: 1
# cpupower -c 1 frequency-info -p
analyzing CPU 1:
  current policy: frequency should be within 1.20 GHz and 2.27 GHz.
                  The governor "userspace" may decide which speed to use
                  within this range.
# cpupower -c 1 frequency-info -f
analyzing CPU 1:
  current CPU frequency: 1199000 (asserted by call to kernel)
# cpupower -c 1 frequency-info --hwlimits
analyzing CPU 1:
  hardware limits: 1.20 GHz - 2.27 GHz

Note: Only the governor userspace allows frequencies to be set.

Standard Operations

All commands seen previously don’t persist after reboot. The standard way to set up CPU governor in a persistent way is through the tuned daemon and the governor directive (alternatively, using rc.local only works if the tuned service is disabled).

If we look at the throughput-performance tuned profile (/usr/lib/tuned/throughput-performance/tuned.conf), we can see:

...
[cpu]
governor=performance
energy_perf_bias=performance
min_perf_pct=100
...

Note: The energy_perf_bias directive allows software on supported Intel processors to more actively contribute to determining the balance between optimum performance and saving power.

Therefore, if you want to define a specific configuration, create a new tuned profile with the tuned inheritance mechanism in the /etc/tuned directory (see the tuned tutorial):

[main]
include=throughput­-performance

[cpu]
...

Behaviour Analysis

To get a better understanding of the way your system behaves, install the powertop package:

# yum install -y powertop

Then, run the powertop command:

# powertop
PowerTOP 2.3      Overview   Idle stats   Frequency stats   Device stats   Tunables            

Summary: 528.4 wakeups/second,  0.0 GPU ops/seconds, 0.0 VFS ops/sec and 4.1% CPU use

                Usage       Events/s    Category       Description
              2.2 ms/s     177.5        Interrupt      PS/2 Touchpad / Keyboard / Mouse
            445.2 us/s      44.4        Timer          tick_sched_timer
              3.4 ms/s      39.2        Interrupt      [27] nvkm
            318.1 us/s      29.2        Timer          hrtimer_wakeup
            137.7 us/s      19.4        Process        [rcu_sched]
...

By pressing the tab key, you get access to different kinds of information (more details are available in the powertop manual):

  • the Overview tab displays a general state of the system.
  • the Idle stats tab presents the CPUs and GPUs currently loaded in the system in relationship with their C-states.
  • the Frequency stats tab shows the P-states of a system in relationship with the idle state.
  • the Device stats tab presents the list of devices in the system that consume the most power.
  • the Tunables tab lists the devices that are present on the system. You can tune the system to be power friendly by toggling each item from bad to good.

Source: RHEL 7 Power Management Guide.

'IBM PowerLinux' 카테고리의 다른 글

Amber off  (0) 2018.07.05
Scale-out LC System Event Log Collection Tool  (0) 2018.06.19
리눅스에서 LVM 구성 및 사용방법  (0) 2018.03.06
LVM home삭제후 root 증설  (0) 2018.02.14
Linux samba 설정  (0) 2017.08.04

 

 

출처 : https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c01847647

 

 

기술 작업 지침

리눅스에서 LVM 구성하는 방법

LVM 은 Logical Volume Manager 의 약자로서, 저장장치들을 좀더 효율적이고 유연하게 관리할 수 있는 커널의 부분과 프로그램을 말합니다. 처음에는 IBM에서 개발되었는데, 그 후에 OSF(현재는 OpenGroup http://www.opengroup.org)에서 차용을 하여 OSF/1 operating system 에서 쓰였습니다. 지금은 HP-UX, Digital Unix operating system, AIX 등의 상용 유닉스에서 쓰고 있습니다. 리눅스 버전은 현재 HP-UX의 것을 모델로 하여 Sistina Software사 (http://www.sistina.com)에서 open source로 개발하고 있습니다. LVM 을 이해하려면 먼저 Software RAID (Redundant Array of Inexpensive Drives)를 언급해야 하는데, 이 둘은 비슷하면서도 큰 차이가 있습니다. 비슷한 점은 여러 물리적인 디스크들을 하나의 논리적인 디스크처럼 다룰 수 있게 함으로서 조합방법에 따라 고용량, 고속, 데이터의 무결성을 실현하는 점입니다. 하지만 분명하게 다른 점이 있는데. lvm은 raid보다 관리 및 확장이 비교적 쉬운 반면, raid에는 lvm에는 없는 disk mirroring (RAID level 1), Parity Stripe (RAID level 4,5) 등의 방식이 있어서 속도 또는 데이터의 무결성을 보장 받을 수 있습니다. 그러나 이런 차이점에도 불구하고 lvm가 주목을 받는 이유는 다음과 같습니다.

  • 쉬운 관리
  • 서로 다른 많은 디바이스 조합 지원
  • 직관적인 저장 장치로의 접근
  • 뛰어난 확장성
  • 믿을만한 안전성과 효율
  • 스냅샷 기능 지원
이와 같은 특징들로 인해 사용자들이 얻는 이득은 많습니다.

VG(Volume Group)은 LVM의 가장 기본적인 요소입니다. 쉽게 말하자면 가상 디스크라고 할 수 있는데, 하나 이상의 실제 물리적으로 존재하는 블록 디바이스가 모여서 VG를 이루게 됩니다.

그 물리적인 블록 디바이스를 PV(Physical Volume)라고 하는데, 거의 대부분의 장치를 PV로 쓸 수 있습니다. 하드디스크 및 그 파티션, 소프트웨어/하드웨어 RAID 장치, 심지어 Loopback 블록 디바이스(파일 시스템상의 파일을 블록 디바이스처럼 쓸 수 있게 해준다) 도 포함합니다.

PV와 대비되는 것이 LV(Logical Volume)입니다. 이것은 가상 파티션이라고도 할 수 있는데, VG를 적당히 나누어 할당한 것이 LV입니다. 사용자는 LV를 일반 디스크나 파티션처럼 쓰면 됩니다. 정리하자면, 하나 이상의 PV가 모여 VG를 이루고, VG를 가상적으로 나누어 할당하면 LV이 됩니다.

  1. PV(Physical Volume) 초기화 작업

    # fdisk /dev/hda

    Command (m for help): t

    Partition number (1-6): 2

    Hex code (type L to list codes): 8e

    Changed system type of partition 2 to 8e (Linux LVM)

    Command (m for help): w



    # fdisk /dev/hdb

    Command (m for help): t

    Partition number (1-6): 2

    Hex code (type L to list codes): 8e

    Changed system type of partition 2 to 8e (Linux LVM)

    Command (m for help): w



  2. PV(Phydical Volume) 생성

    # pvcreate /dev/hda2

    pvcreate -- physical volume “/dev/hda2”

    successfully created



    # pvcreate /dev/hdb2

    pvcreate -- physical volume “/dev/hdb2”

    successfully created



  3. VG(Volume Group) 생성

    # vgcreate -s 16m new_vg /dev/hda /dev/hdb

    위의 명령은 두 개의 스크를 합쳐서 new_vg라는 VG를 만든다는 것이다.

    옵션 ‘-s 16m’은 PE(Physical Extent)의 크기를 16MB로 정하는 것인데, 만약 옵션 ‘-s’가 생략되면 기본적으로 PE의 크기는 4MB가 된다. VG가 만들어지면 /dev/VolumeGroupName라는 형식으로 디렉토리가 생기게 된다.

    위의 예에서는 /dev/new_vg 가 될 것이다.



  4. VG(Volume Group)명 변경 또는 삭제방법

    1. 변경방법

      # vgrename new_vg vg00

    2. 삭제방법

      VG에 LV가 남아있지 않고 비활성화되어 있으면 vgremove로 지울 수 있습니다. 비활성화 시키는 방법은, # vgchange -a n vg00 이고, 다음의 명령으로 지워집니다.

      # vgremove vg00



  5. VG(Volume Group) 확장방법

    VG의 확장과 축소는 PV의 추가, 제거로 이루어집니다.

    VG에 PV를 추가하는 명령은 vgextend인데, vg00이라는 VG에 PV인 /dev/hda2를 추가하려면 다음과 같이 실행해야 합니다.

    # vgextend vg00 /dev/hda2



  6. VG(Volume Group) 축소방법

    PV를 VG에서 제거하는 명령은 vgreduce입니다. 그러나 PV에 PE가 할당되어 있으면 제거되지 않습니다. 그 할당된 PE를 다른 PV로 옮겨야 제거될 것입다. 빈 PV인 /dev/hda2를 vg00에서 제거하려면,

    # vgreduce vg00 /dev/hda2



  7. LV(Logical Volume) 생성

    VG에 LV를 만들면 /dev/VolumeGroupName/LogicalVolumeName의 형식으로 블록 디바이스가 만들어집니다. 이것은 일반 블록 디바이스와 다를 바 없는 용도로 쓸 수 있습니다. LV 를 만드는 명령은 lvcreate인데 일반적인 예는 다음과 같습니다.

    # lvcreate -L 4g -n data vg00

    위의 명령은 4GB 크기로 vg00이라는 VG에 data라는 새로운 LV를 만드는 것입니다. 옵션 ‘-L’은 LV의 크기를 지정하는 것인데, 숫자 뒤의 접미사에 따라 숫자의 단위가 달라집니다. k는 KB, m은 MB, g는 GB, t는 TB를 뜻합니다. 만약, 접미사가 안 붙어있으면 숫자의 단위는 MB가 됩니다. ‘-L’ 대신 ‘-l’를 쓸 수도 있는데, ‘-l’ 뒤에는 LE의 갯수를 쓰면 됩니다.



  8. LV(Logical Volume)명 변경 및 삭제방법
    1. 변경방법

      # lvrename /dev/vg00/data /dev/vg00/db

    2. 삭제방법

      # lvremove /dev/vg00/db



  9. LV(Logical Volume) 확장방법

    # lvextend -L +2g /dev/vg00/db

    위의 명령은 /dev/vg00/db의 크기를 2GB 만큼 늘립니다. 기존의 크기가 4GB 였다면, 옵션 ‘-L 6g’를 줬어도 같은 결과를 보일 것입니다. 역시 여기에서도 옵션 ‘-l’를 쓰면 LE 단위로 늘릴 수 있습니다.



    다음은 일반적인 ext2 의 확장 절차입니다.

    • # umount /var/lib/mysql
    • # lvextend -L +10g /dev/databases/mysql
    • # resize2fs /dev/databases/mysql
    • # mount /dev/databases/mysql /var/lib/mysql
    NOTE: reiserfs : reiserfs는 resize_reiserfs 라는 명령으로 크기를 바꿀 수 있습니다.

    마운트된 상태에서도 크기를 늘릴 수 있는데, 그 예는 다음과 같습니다.

    # lvextend -L +5g /dev/devel/cvs

    # resize_reiserfs -f /dev/devel/cvs

    언마운트해서 늘리려면 다음 같이 합니다.

    # umount /var/cvs

    # lvextend -L +5g /dev/devel/cvs

    # resize_reiserfs -f /dev/devel/cvs

    # mount /dev/devel/cvs /var/cvs



  10. LV(Logical Volume) 축소방법
    • ext2 : e2fsadm 을 쓰는 것이 간단합니다.

      # umount /home

      # lvreduce -L -2g /dev/vg00/home 또는

      # lvresize -L -2g /dev/vg00/home

      # mount /dev/vg00/home /home

    • reiserfs : 크기를 늘릴 때와는 달리 마운트 된 상태에서는 줄일 수 없습니다

      # umount /usr

      # resize_reiserfs -s -1g /dev/vg00/usr

      # lvreduce -L -1g /dev/vg00/usr

      # mount /dev/vg00/usr /usr



  11. LVM 설정의 Backup 방법

    먼저 설정 백업 파일의 내용을 보려면 다음과 같이 합니다.

    # vgcfgrestore -f /etc/lvm/backup/vg00 -ll vg00



    이 파일을 이용해 설정을 복구하려면 다음과 같이 합니다.

    # vgchange -a n vg00

    # vgcfgrestore -f /etc/lvm/backup/vg00 vg00

    # vgchange -a y vg00



  12. 두개의 ide disk를 하나의 striped mapping LV로 묶는 방법

    80GB 짜리(실제는 약 76.1GB) IDE 하드디스크 두 개를 striped mapping LV 로 묶는 과정입니다. 먼저 블록 디바이스 이름이 /dev/hde, /dev/hdf 인 디스크들을 PV로 만듭니다.

    # pvcreate /dev/hde /dev/hdf



    이렇게 만든 PV들을 합쳐서 VG를 만듭니다.

    # vgcreate vg00 /dev/hde /dev/hdf



    이제 VG에 LV를 만듭니다.

    # lvcreate -i 2 -I 4 -L 152g -n striped_lv vg00

    위의 명령으로 152GB 짜리 striped mapping LV가 만들어진 것입니다. 옵션을 설명하면 ‘-i 2’는 두 개의 PV를 striped 하는데 쓰고, ‘-I 4’는 PE를 4KB로 쪼개서 PV들에게 저장한다는 것입니다. ‘-I’ 옵션에는 2n (n은 0과 7사이의 숫자), 즉 1, 2, 4, 8, 16, 32, 64, 128 의 숫자를 쓸 수 있습니다



    파일 시스템을 만들고, 마운트를 해서,

    # mkreiserfs /dev/vg00/striped_lv 또는

    # mke2fs –j /dev/vg00/striped_lv

    # mount /dev/vg00/striped_lv /mnt

  13. 새로운 디스크로 PV 교체방법

    환경은 /dev/hdd, /dev/sda, /dev/sdb가 vg01에 속해있는데, 그중에서 /dev/hdd를 빼고 /dev/sdc를 더하는 과정입니다.

    먼저 /dev/sdc 를 PV 로 만들고,

    # pvcreate /dev/sdc



    /dev/sdc를 vg01에 추가한 후에,

    # vgextend vg01 /dev/sdc



    /dev/hdd의 PE 들을 /dev/sdc로 옮깁니다.

    # pvmove /dev/hdd /dev/sdc



    성공적으로 옮겨졌으면 vg01에서 /dev/hdd를 제거합니다.

    # vgreduce vg01 /dev/hdd

  14. snapshot LV를 만들어서 안전한 백업방법

    데이터의 변동이 많은 /var등의 디렉토리는 백업 도중에 데이터가 엉키는 수가 있습니다. 그것을 방지하기 위해서, snapshot LV를 만들어 데이터를 고정시킨 후 백업을 하면 됩니다. 일단 snapshot LV 를 만듭니다.(한 줄로)

    # lvcreate -s -L 5g -n mysqlbackup /dev/vg01/mysql

    위의 명령은 /dev/vg01/mysql에 대한 snapshot LV인 /dev/vg01/mysqlbackup를 만듭니다. 옵션 ‘-s’는 만들어질 LV가 snapshot 형식임을 정해주고 있습니다. 옵션 ‘-L’ 는 LV 의 최대 크기를 정해줍니다. 대개 원본 LV와 크기를 같게 하는 것이 좋습니다. 사실 snapshot LV는 원본이 갱신되는 부분만 고정시키려고 데이터를 가져와 LE에 할당하기 때문에 많은 LE가 필요하지 않습니다.



    이제 만든 LV 를 마운트 시킨 후

    # mount /dev/vg01/mysqlbackup /mnt



    백업을 하면 된다.

    # tar cf /dev/rmt0 /mnt



    백업을 마쳤으면 언마운트시킨 후에 snapshot LV 을 지웁니다.

    # umount /mnt

    # lvremove /dev/vg01/mysqlbackup

LVM관련 디스크가 변경되었을 경우

# vgscan –v –mknodes 를 실행하면

vgscan scans all SCSI, (E)IDE disks, multiple devices and a bunch of other disk devices in the system looking for LVM physical volumes and volume groups. Define a filter in lvm.conf(8) to restrict the scan to avoid a CD ROM, for example.

In LVM2, vgscans take place automatically; but you might still need to run one explicitly after changing hardware.

See lvm for common options

--mknodes

Also checks the LVM special files in /dev that are needed for active logical volumes and creates any missing ones and removes unused ones.

'IBM PowerLinux' 카테고리의 다른 글

Scale-out LC System Event Log Collection Tool  (0) 2018.06.19
RHEL7: How to get started with CPU governor  (0) 2018.03.20
LVM home삭제후 root 증설  (0) 2018.02.14
Linux samba 설정  (0) 2017.08.04
Linux Network SPEED test iperf  (0) 2017.07.09

 

# lvremove /dev/mapper/vg_vpsny23-lv_home
# lvextend -l +100%FREE -r /dev/mapper/vg_vpsny23-lv_root

'IBM PowerLinux' 카테고리의 다른 글

RHEL7: How to get started with CPU governor  (0) 2018.03.20
리눅스에서 LVM 구성 및 사용방법  (0) 2018.03.06
Linux samba 설정  (0) 2017.08.04
Linux Network SPEED test iperf  (0) 2017.07.09
리눅스 Filesystem cache 정책 변경  (0) 2017.02.21

 

 참고 사이트


http://hwengineer.blogspot.com

http://shareithw.blogspot.kr

http://leearmykr.blogspot.com/






Procedure

  1. Download the ISO file for the Linux distribution and create a bootable USB device or DVD.
  2. Insert your USB device into the front USB port.
  3. Verify that USB device is shown as an option in Petitboot as USB: sdc:
    USB: sdc /
    		Rescue mode
    		Installcopy to clipboard
    Note:Select Rescan devices if USB device does not appear.

    If you are installing Red Hat Enterprise Linux 7.2, you must provide some additional details to the installer. Follow these steps:

    1. Record the UUID of the USB device. For example, the UUID of the USB device in the following example is 2015-10-30-11-05-03-00.
      [USB: sdb1 / 2015-10-30-11-05-03-00]
                  Rescue a Red Hat Enterprise Linux system (64-bit kernel)
                  Test this media & install Red Hat Enterprise Linux 7.2  (64-bit kernel)
               *  Install Red Hat Enterprise Linux 7.2 (64-bit kernel)copy to clipboard
    2. Select Install Red Hat Enterprise Linux 7.2 (64-bit kernel) and press e (Edit) to open the Petitboot Option Editor window.
    3. Move the cursor to the Boot arguments section and add the following information:
      inst.stage2=hd:UUID=your_UUIDcopy to clipboard
      where your_UUID is the UUID that you recorded.
      Petitboot Option Editor
                    qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
      
                    Device:    ( ) sda2 [f8437496-78b8-4b11-9847-bb2d8b9f7cbd]
                                    (*) sdb1 [2015-10-30-11-05-03-00]
                                    ( ) Specify paths/URLs manually
      
                    Kernel:         /ppc/ppc64/vmlinuz
                    Initrd:         /ppc/ppc64/initrd.img
                    Device tree:
                    Boot arguments: ro inst.stage2=hd:UUID=2015-10-30-11-05-03-00    
      
                       [    OK    ]  [   Help   ]  [  Cancel  ]copy to clipboard
    4. Select OK to save your options and return to the Main menu.
  4. Select Install.
    Note:Be patient! It can sometimes take a couple minutes for the installation to begin.
  5. Follow the installation wizard for your Linux distribution to set up disk options, your user name and password, time zones, and so on. The last step is to restart your system.
  6. After the system restarts, Petitboot displays the option to boot the Linux distribution that you installed. Select this option and press Enter.

 




OS 환경 구성

 

 ● Selinux disable

 

    현재 상태 확인

[root@ac922 ~]# getenforce

Enforcing

è  Enforcingenable 상태로 보안 적용이 되어 있는 상태

 

    Selinux를 임시로 disable 적용

[root@ac922 ~]# setenforce 0

[root@ac922 ~]# getenforce

Permissive

è  Setenforce 명령어로 현재 적용되어 있는 selinuxdisable OS 재부팅시 enable로 변경됨 (임시 적용)

 

    Selinux up되지 않도록 설정

[root@ac922 ~]# vi /etc/selinux/config

 

SELINUX=enforcing

SELINUX=disabled

로 변경 후 저장

è  OS 부팅시 selinux를 사용하지 않게 변경

 

 

 ● Firewall (iptables) disable

 

 

    Firewall service 상태 확인

[root@ac922 ~]# systemctl status firewalld.service

firewalld.service - firewalld - dynamic firewall daemon

   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)

   Active: active (running) since Thu 2018-02-22 15:40:27 KST; 19h ago

     Docs: man:firewalld(1)

 Main PID: 2012 (firewalld)

   CGroup: /system.slice/firewalld.service

           └─2012 /usr/bin/python -Es /usr/sbin/firewalld --nofork –nopid

 

è  Active (running) 상태면 firewalld.service 를 내려야 함

 

    Firewall service shutdown

[root@ac922 ~]# systemctl disable firewalld.service

Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.

Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

 

è  OS 재부팅시 firewall (iptables) 서비스가 실행되지 않게 변경

 

    Firewall service shutdown

 

[root@ac922 ~]# iptables –F

è  Firewall 설정을 clear

[root@ac922 ~]# iptables –L

è  Firewall 적용내역 확인

 

 

 

 ● 인터넷이 없는 환경에서 Local repository 구성방법

 

 

    CD-ROM 을 이용한 local repository 구성

 

[root@ac922 ~]# mkdir /cdrom

[root@ac922 ~]# mount -t iso9660 /dev/sr0 /cdrom

è  OS 설치 CDROM/cdrommount

 

[root@ac922 ~]# cd /etc/yum.repos.d/

[root@ac922 yum.repos.d]# vi local.repo

 

[root@ac922 yum.repos.d]# cat local.repo

[local-repository]

name=local-repository

baseurl=file:///cdrom/

gpgcheck=0

 

    YUMrepository meta-db 업데이트

 

[root@ac922 yum.repos.d]# yum update

Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager

This system is not registered with an entitlement server. You can use subscription-manager to register.

local-repository                                                                                    | 4.1 kB  00:00:00

(1/2): local-repository/group_gz                                                                    | 129 kB  00:00:00

(2/2): local-repository/primary_db                                                                  | 3.2 MB  00:00:00

No packages marked for update

 

è  YUM repository update 실시

 

 

 

 

 

 

 ● Redhat epel 설치

-       Extra Package for Enterprise Linux repository configuration

-       기본적으로 Redhat(CentOS) 제공하는 패키지 외 extra package 사용을 원할 때 epel-release를 설치 

 

[root@ac922 ~]# wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

 

[root@ac922 ~]# rpm -ihv epel-release-latest-7.noarch.rpm

 

[root@ac922 ~]# yum update

 

 

[root@ac922 ~]# yum install nmon

Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager

This system is not registered with an entitlement server. You can use subscription-manager to register.

Resolving Dependencies

--> Running transaction check

---> Package nmon.ppc64le 0:16g-3.el7 will be installed

--> Finished Dependency Resolution

 

Dependencies Resolved

 

===========================================================================================================================

 Package                    Arch                          Version                         Repository                  Size

===========================================================================================================================

Installing:

 nmon                       ppc64le                       16g-3.el7                       epel                        70 k

 

Transaction Summary

===========================================================================================================================

Install  1 Package

 

Total download size: 70 k

Installed size: 199 k

Is this ok [y/d/N]: y

 

è  테스트용 nmon 패키지 설치 

 

 

 

 

 

 ●  Redhat subscription enable


-> 당장 subscription 이 없을때 redhat.com 에서 30일 체험판 신청 후 subscrtion 등록 하면 30일간은 사용가능

 




### Operating System and Repository Setup


   1. Enable 'optional' and 'extra' repo channels


          $ sudo subscription-manager repos --enable=rhel-7-for-power-9-optional-rpms

          $ sudo subscription-manager repos --enable=rhel-7-for-power-9-extras-rpms


   2. Install packages needed for the installation


          $ sudo yum -y install wget nano bzip2


   3. Enable EPEL repo ( 위에 설치함 생략 )


          $ wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

          $ sudo rpm -ihv epel-release-latest-7.noarch.rpm


   4. Load the latest kernel


          $ sudo yum update kernel kernel-tools kernel-tools-libs kernel-bootwrapper

          $ reboot # This reboot may be deferred until after the NVIDIA steps below.


      Or do a full update


          $ sudo yum update

          $ sudo reboot # This reboot may be deferred until after the NVIDIA steps below.

 


 

 ● Power9 nvidia driver 설정




### NVIDIA Components


Before installing the NVIDIA components the udev Memory Auto-Onlining Rule must

be disabled for the CUDA driver to function properly. To disable it:


   1. Edit the /lib/udev/rules.d/40-redhat.rules file.


          $ sudo nano /lib/udev/rules.d/40-redhat.rules


   2. Comment out the following line and save the change:


          SUBSYSTEM=="memory", ACTION=="add", PROGRAM="/bin/uname -p", RESULT!="s390*", ATTR{state}=="offline", ATTR{state}="online"


   3. Reboot the system for the changes to take effect.


          $ sudo reboot

 




The Deep Learning packages require CUDA, cuDNN, and GPU driver packages

from NVIDIA.


The required and recommended versions of these components are:


    | Component    | Required | Recommended |

    |--------------|----------|-------------|

    | CUDA Toolkit | 9.1      | 9.1.85      |

    | cuDNN        | 7.0.5    | 7.0.5       |

    | GPU Driver   | 387.36   | 387.36      |



-------------------------------------------------------------------------------------------------------------------------

 ● CUDA 9.1 & Cudnn 설치

-> cudnn은 압축 풀어서 덮어쓰기 만 하면 설치 된것 ( 따로 패키지 설치 x )



   1. Download and install NVIDIA CUDA 9.1 from [https://developer.nvidia.com/cuda-downloads](https://developer.nvidia.com/cuda-downloads)

      - Select *Operating System:* **Linux**

      - Select *Architecture:* **ppc64le**

      - Select *Distribution* **RHEL**

      - Select *Version* **7**

      - Select the *Installer Type* that best fits your needs

      - Follow the **Linux POWER9** installation instructions in the *CUDA Quick

        Start Guide* (linked from

        [https://developer.nvidia.com/cuda-downloads](https://developer.nvidia.com/cuda-downloads)),

        including the steps describing how to set up the CUDA development

        environment by updating `PATH` and `LD_LIBRARY_PATH`.


   2. Download NVIDIA cuDNN 7.0.5 for CUDA 9.1 from

      [https://developer.nvidia.com/cudnn](https://developer.nvidia.com/cudnn)

      (Registration in NVIDIA's Accelerated Computing Developer Program is required)


      - cuDNN v7.0.5 Library for Linux (Power8/Power9)


   3. Install the cuDNN v7.0 packages


          $ sudo tar -C /usr/local --no-same-owner -xzvf cudnn-9.1-linux-ppc64le-v7.0.5.tgz


 


 ● Anaconda 설치

- anaconda 2 와 anaconda 3 모두 설치 (아래는 anaconda2 설치 예시 - anaconda3 도 동일한 절차로 설치)
- 설치시 default 설치 위치만 변경 권장 "/root/anaconda2" 에서 "/opt/anaconda2" 로 PATH 변경



A number of the Deep Learning frameworks require Anaconda. Anaconda is a platform-agnostic data science distribution with a collection of 1,000+ open source packages with free community support.


Download and Install Anaconda. Installation requires input for license agreement, install location (default is `$HOME/anaconda2`) and permission to modify the `PATH` environment variable (via `.bashrc`).


        $ wget https://repo.continuum.io/archive/Anaconda2-5.0.0-Linux-ppc64le.sh

        $ bash Anaconda2-5.0.0-Linux-ppc64le.sh

        $ source ~/.bashrc

 



 


 ● Installing the Deep Learning Frameworks ( PowerAI 설치 )


### IBM Spectrum MPI Install



Download IBM Spectrum MPI from the ESP download site.


   1. Install the rpms

   

          $ sudo rpm -ihv ibm_smpi_lic_s-10.02.00*.ppc64le.rpm ibm_smpi-10.02.00*.ppc64le.rpm


### Software Repository Setup


IBM TensorFlow ESP for Power AC922 Deep Learning packages are distributed in an rpm file and is available from the ESP download site.

Installing the rpm creates an installation repository on the local machine.


   1. Install the repository package:


          $ sudo rpm -ihv mldl-repo-local*.rpm


### Installing all frameworks at once


All the Deep Learning frameworks can be installed at once using the

`power-mldl` meta-package:


        $ sudo yum install power-mldl-esp



### Installing frameworks individually  

 -> 위에 power-mldl-esp 로 설치 했으면 SKIP!!!!!!


The Deep Learning frameworks can be installed individually if preferred.

The framework packages are:


   - `tensorflow` - Google TensorFlow, v1.4.0

   - `tensorboard` - Web Applications for inspecting TensorFlow runs and graphs, v0.4.0rc3

   - `ddl-tensorflow` - Distributed Deep Learning custom operator for TensorFlow


Each can be installed with:


        $ sudo yum install <framework>-cuda9.1


### Accept the License Agreement


Read the license agreements and accept the terms and conditions before using Spectrum MPI or any of the frameworks.


        $ sudo IBM_SPECTRUM_MPI_LICENSE_ACCEPT=no /opt/ibm/spectrum_mpi/lap_se/bin/accept_spectrum_mpi_license.sh

        $ sudo /opt/DL/license/bin/accept-powerai-license.sh


After reading the license agreements, future installs may be automated to silently accept the license agreements.


        $ sudo IBM_SPECTRUM_MPI_LICENSE_ACCEPT=yes /opt/ibm/spectrum_mpi/lap_se/bin/accept_spectrum_mpi_license.sh

        $ sudo IBM_POWERAI_LICENSE_ACCEPT=yes /opt/DL/license/bin/accept-powerai-license.sh


 


 ● PowerAI 환경을 위한 OS 튜닝 




## Tuning Recommendations


Recommended settings for optimal Deep Learning performance on the IBM Power System AC922 are:


   - Enable Performance Governor


         $ sudo yum install kernel-tools

         $ sudo cpupower -c all frequency-set -g performance


   - Enable GPU persistence mode

      

         $ sudo systemctl enable nvidia-persistenced

         $ sudo systemctl start nvidia-persistenced


   - For TensorFlow, set the SMT mode


         $ sudo ppc64_cpu --smt=4


   - For TensorFlow with DDL, set the SMT mode


         $ sudo ppc64_cpu --smt=2



## Getting Started with MLDL Frameworks


### General Setup


Most of the PowerAI packages install outside the normal system search

paths (to `/opt/DL/...`), so each framework package provides a shell

script to simplify environmental setup (e.g. `PATH`, `LD_LIBRARY_PATH`,

`PYTHONPATH`).


We recommend users update their shell rc file (e.g. `.bashrc`) to source

the desired setup scripts. For example:


    $ source /opt/DL/<framework>/bin/<framework>-activate


Each framework also provides a test script to verify basic function:


    $ <framework>-test


### Note about dependencies


A number of the PowerAI frameworks (for example, TensorFlow, and TensorBoard)

have their dependencies satisfied via Anaconda packages.  These dependencies are validated

by the `<framework>-activate` script to ensure they are installed and, if not, the script will fail.


For these frameworks, the `/opt/DL/<framework>/bin/install_dependencies` script must be run

prior to activation to install the required packages.


For example:


    $ source /opt/DL/tensorflow/bin/tensorflow-activate

    Missing dependencies ['backports.weakref', 'mock', 'protobuf']

    Run "/opt/DL/tensorflow/bin/install_dependencies" to resolve this problem.


    $ /opt/DL/tensorflow/bin/install_dependencies

    Fetching package metadata ...........

    Solving package specifications: .


    Package plan for installation in environment /home/rhel/anaconda2:


    The following NEW packages will be INSTALLED:


        backports.weakref: 1.0rc1-py27_0

        libprotobuf:       3.4.0-hd26fab5_0

        mock:              2.0.0-py27_0

        pbr:               1.10.0-py27_0

        protobuf:          3.4.0-py27h7448ec6_0


    Proceed ([y]/n)? y


    libprotobuf-3. 100% |###############################| Time: 0:00:02   2.04 MB/s

    backports.weak 100% |###############################| Time: 0:00:00  12.83 MB/s

    protobuf-3.4.0 100% |###############################| Time: 0:00:00   2.20 MB/s

    pbr-1.10.0-py2 100% |###############################| Time: 0:00:00   3.35 MB/s

    mock-2.0.0-py2 100% |###############################| Time: 0:00:00   3.26 MB/s


    $ source /opt/DL/tensorflow/bin/tensorflow-activate

    $


 




 ● Getting Started with Tensorflow

- TF 기본 테스트



The TensorFlow homepage

([https://www.tensorflow.org/](https://www.tensorflow.org/)) has a

variety of information, including Tutorials, How Tos, and a Getting

Started guide.


Additional tutorials and examples are available from the community, for

example:


   - [https://github.com/nlintz/TensorFlow-Tutorials](https://github.com/nlintz/TensorFlow-Tutorials)

   - [https://github.com/aymericdamien/TensorFlow-Examples](https://github.com/aymericdamien/TensorFlow-Examples)


#### Distributed Deep Learning (DDL) Custom Operator for TensorFlow


IBM TensorFlow ESP for Power AC922 includes a Technology Preview of the

IBM PowerAI Distributed Deep Learning (DDL) custom operator for TensorFlow.

The DDL custom operator uses IBM Spectrum MPI and NCCL to provide high-speed

communications for distributed TensorFlow.


The DDL custom operator can be found in the `ddl-tensorflow` package.

For more information about DDL and about the TensorFlow operator, see:


   - `/opt/DL/ddl-doc/doc/README.md`

   - `/opt/DL/ddl-tensorflow/doc/README.md`

   - `/opt/DL/ddl-tensorflow/doc/README-API.md`


The DDL TensorFlow operator makes it easy to enable models

for distribution. The package includes examples of models enabled

with DDL including TensorFlow High Performance and Slim models:


    $ source /opt/DL/ddl-tensorflow/bin/ddl-tensorflow-activate


    $ ddl-tensorflow-install-samples <somedir>


The Slim model examples are based on a specific commit of the TensorFlow models

repo with a small adjustment. If you prefer to work from an upstream clone,

rather than the packaged examples:


    $ git clone https://github.com/tensorflow/models.git

    $ cd models

    $ git checkout 11883ec6461afe961def44221486053a59f90a1b

    $ git revert fc7342bf047ec5fc7a707202adaf108661bd373d

    $ cp /opt/DL/ddl-tensorflow/examples/slim/train_image_classifier.py slim/


#### Additional TensorFlow Features


The PowerAI TensorFlow packages include TensorBoard. See:

[https://www.tensorflow.org/get_started/summaries_and_tensorboard](https://www.tensorflow.org/get_started/summaries_and_tensorboard)


The TensorFlow 1.4.0 package includes support for additional features:


   - HDFS

   - NCCL

   - experimental XLA JIT compilation

     (see [https://www.tensorflow.org/performance/xla/](https://www.tensorflow.org/performance/xla/))


## Uninstalling MLDL Frameworks


The MLDL Framework packages can be uninstalled individually the same way they were installed. In order to uninstall all MLDL packages and the repo used to install them run:


    $ sudo yum remove powerai-license

    $ sudo yum remove mldl-repo-local-esp


 



 ●  Docker / Nvidia-Docker 설치


참고 -> http://hwengineer.blogspot.com/2018/02/ppc64le-docker-nvidia-docker-repository.html



1. Redhat 환경의 경우

[root@ac922 ~]# cat /etc/yum.repos.d/docker.repo
[docker]
name=Docker
baseurl=http://ftp.unicamp.br/pub/ppc64el/rhel/7/docker-ppc64el/
enabled=1
gpgcheck=0

 

# systemctl enable docker.service
# systemctl start docker.service
-> Docker 서비스 시작!

 

# docker images

-> Docker 실행 확인

 

///////////////////////////////////////////////////////////////////////////////

 

[이하 nvidia-docker]


RHEL-based distributions


# distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
# curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.repo | sudo tee /etc/yum.repos.d/nvidia-docker.repo

-> Nvidia-docker repository 업데이트

 

# yum update

# yum install nvidia-docker

 

# nvidia-docker

-> nvidia-docker 설치 확인

 

# /usr/bin/nvidia-docker-plugin &

-> nvidia-docker 사용을 위해 plugin 을 background로 실행

-> 재부팅후에도 background 로 실행하기 위해 rc.local 에 등록

 

# vi /etc/rc.d/rc.local

/usr/bin/nvidia-docker-plugin &

추가해줌

 

# chmod +x /etc/rc.d/rc.local

-> rc.local에 실행권한 필수

 

 

 

 


 ● TF 1.5 버전 설치 시 참조


http://hwengineer.blogspot.com/2018/04/ac922-redhat-74-python-36-tensorflow.html


 ●  AC922 에서 nvidia-smi 시 unknown 에러 발생시 추가 작업 및 설정 확인


https://hwengineer.blogspot.com/2018/04/ac922-cuda-91.html



 ●  인터넷 안되는 환경에서 -> Redhat Subscrtion-manager DISABLE !!!!!!!!!!!!!!!!!!!!!!!!



Disabling the Subscription-Manager Repository

When a system is registered using Subscription-Manager, the rhsmcertd process creates a special yum repository — redhat.repo. As “Enabling Supplementary and Optional Repositories” describes, as the system adds subscriptions, the product channels are added to the redhat.repo file.


Maintaining a redhat.repo file may not be desirable in some environments. It can create static in content management operations if that repository is not the one actually used for subscriptions, such as for a disconnected system or a system using a local content mirror.


This default redhat.repo repository can be disabled by editing the Subscription-Manager configuration and setting the manage_repos value to zero (0).


Raw

[root@server ~]# subscription-manager config --rhsm.manage_repos=0 



///////////// 이하 테스트  ///////////////

 

 

 

newwell 설치 정리

 

- Extra Packages for Enterprise Linux (EPEL)
-> 추가 패키지 repository 등록

 

redhat7 버전 repository 설치

# wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
# rpm -ivh epel-release-latest-7.noarch.rpm
# yum update

# yum repolist




- cuda 9.1

# rpm -ivh cuda-repo-ubuntu1604-9-0-local_9.0.176-1_ppc64el.deb

# yum install cuda

 


- cudnn 7.0 for cuda 9.1

# wget http://developer2.download.nvidia.com/compute/machine-learning/cudnn/secure/v7.0.5/prod/9.1_20171129/cudnn-9.1-linux-ppc64le-v7.tgz?9GVxLevnEbiZ58fRLwXMF4dcgjWPoUHm1vfRDm_87tF5yDIjNeOyAV5vZwaygOrMjgXlVlAeEPaB9CL2oPbggLw08gUYN8xq62eGOwbacmvE9X7Lyvdp7_yqzQQCMfyfGjHH40qyjLlMwt3l4CypdNdCtw4XyBRQdpOdUI8k5eAylpHnPnngkIcE9-ReD70rYBM50Oi75p75itEl

# mv http://developer2.download.nvidia.com/compute/machine-learning/cudnn/secure/v7.0.5/prod/9.1_20171129/cudnn-9.1-linux-ppc64le-v7.tgz?9GVxLevnEbiZ58fRLwXMF4dcgjWPoUHm1vfRDm_87tF5yDIjNeOyAV5vZwaygOrMjgXlVlAeEPaB9CL2oPbggLw08gUYN8xq62eGOwbacmvE9X7Lyvdp7_yqzQQCMfyfGjHH40qyjLlMwt3l4CypdNdCtw4XyBRQdpOdUI8k5eAylpHnPnngkIcE9-ReD70rYBM50Oi75p75itEl cudnn-9.1-linux-ppc64le-v7.solitairetheme8

# tar -xf cudnn-9.1-linux-ppc64le-v7.solitairetheme8

cd cuda/tagets/ppc64le-linux
# scp -rp include/ /usr/local/cuda-9.1/targets/ppc64le-linux/
# scp -rp lib/ /usr/local/cuda-9.1/targets/ppc64le-linux/

 

- Bazel 9.0 설치

# unzip bazel-0.9.0-dist.zip

# yum install *jdk*

# compile.sh

# scp -rp output/bazel /usr/local/bin/


- protobuf 설치

yum install autoconf automake libtool

git clone https://github.com/google/protobuf

 

- Tensorflow 1.4

# yum install -y git patch python-pip python-wheel numpy

# git clone --recurse-submodules https://github.com/tensorflow/tensorflow
# cd tensorflow
# git checkout master
# ./configure

# bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


<초기환경 설정>
# apt-get install make automake autoconf gfortran-4.8-multilib-powerpc-linux-gnu libgfortran-4.8-dev-powerpc-cross gfortran libgfortran-4.8-dev gcc-powerpc-linux-gnu g++ pkg-config cmake-curses-gui zlib1g-dev zip unzip libtool openjdk-8-jdk python python-all-dev swig python-numpy g++-4.8 g++-4.8-multilib-powerpc-linux-gnu zlibc re2c libre2-dev libcupti-dev git

<IBM 전용>
-Advanced Toolchain

 
wget ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu/dists/xenial/at11.0/binary-ppc64el/advance-toolchain-at11.0-mcore-libs_11.0-2_ppc64el.deb &

wget ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu/dists/xenial/at11.0/binary-ppc64el/advance-toolchain-devel_11.0-2_ppc64el.deb &
wget ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu/dists/xenial/at11.0/binary-ppc64el/advance-toolchain-perf_11.0-2_ppc64el.deb &
wget ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu/dists/xenial/at11.0/binary-ppc64el/advance-toolchain-runtime_11.0-2_ppc64el.deb &
-> 파일 4개 다운로드

dpkg -i advance-toolchain-at11.0-runtime
dpkg -i advance-toolchain-at11.0-perf_11.0-2_ppc64el.deb
dpkg -i advance-toolchain-at11.0-devel_11.0-2_ppc64el.deb
dpkg -i advance-toolchain-at11.0-mcore-libs_11.0-2_ppc64el.deb
-> 해당파일 4개 설치 후 "/opt/at11.0/bin/" 디렉토리내 파일 사용

example) advanced toolchain을 설치하면 gcc 최신버전이 설치됨

root@ubuntu:~# gcc --version
gcc (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.5) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

root@ubuntu:~# /opt/at11.0/bin/gcc --version
gcc (GCC) 7.2.1 20171116 (Advance-Toolchain-at11.0) [ibm/gcc-7-branch revision 254526]
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.





<개발툴>

-CUDA 9.1

 

root@ubuntu:/data/test# dpkg -i cuda-repo-ubuntu1604-9-1-local_9.1.85-1_ppc64el.deb
root@ubuntu:/data/test# apt-get update
W: GPG error: file:/var/cuda-repo-9-1-local  Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY F60F4B3D7FA2AF80
W: There is no public key available for the following key IDs:
F60F4B3D7FA2AF80 
root@ubuntu:/etc/apt/sources.list.d# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys  F60F4B3D7FA2AF80
Executing: /tmp/tmp.K9QlvbZhFq/gpg.1.sh --keyserver
keyserver.ubuntu.com
--recv-keys
F60F4B3D7FA2AF80
# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys  F60F4B3D7FA2AF80
# grep -r --include '*.list' '^deb ' /etc/apt/sources.list*
-> Repository list 확인
# apt install cuda



-CuDNN 7

 

tar -xf cudnn-9.1-linux-ppc64le-v7.solitairetheme8
cp -rp cuda/targets/ppc64le-linux/include/cudnn.h /usr/local/cuda-9.1/include
cp -rp cuda/targets/ppc64le-linux/lib/* /usr/local/cuda-9.1/lib64/
-Anaconda (python 3.6)
wget https://repo.continuum.io/archive/Anaconda3-5.0.0-Linux-ppc64le.sh
chmod 755 Anaconda3-5.0.0-Linux-ppc64le.sh
./Anaconda3-5.0.0-Linux-ppc64le.sh
conda install bazel numpy
-> bazel numpy 라이브러리 설치
///
apt install software-properties-common



 

 

- Bazel 최신버전 설치

https://docs.bazel.build/versions/master/install-compile-source.html

# wget https://github.com/bazelbuild/bazel/releases/download/0.9.0/bazel-0.9.0-dist.zip
# unzip bazel-0.9.0-dist.zip

# ./compile.sh
# bazel version
-> Version 확인

# cp -rp output/bazel /usr/local/bin/

 


-tensorflow 1.4

 

 

# git clone --recurse-submodules https://github.com/tensorflow/tensorflow
# cd tensorflow
# git checkout master
# ./configure


 


-Spyder 3.2.4


-jupyter
-Keras 2.0
 
<라이브러리>
-numpy
-scipy
-scikit learn 0.19.1
-matplotlib 2.1.1
-pydot_ng
-graphviz
-Pillow
-OpenCV
 
<원격접속>
-xrdp-xvnc setting (.xsession-->mate-session)
-ftp setting

////////////////////////////





'IBM PowerAI' 카테고리의 다른 글

docker 도커 설명  (0) 2017.07.09
spectrum scale 구성 (gpfs)  (0) 2017.07.07
관련 자료  (0) 2017.07.06

 

CentOS7에서 nmcli를 사용하여 본딩 설정을 할 수 있습니다.

이전

  • 네트워크 워크의 페이스 북이 2 つ ある
  • 양방향으로도 스위치 연결이되어있는
  • 片方 だけ IP 주소가 설정되어있어 유효하게 된 것

↓ こ ん な 환경

$ nmcli c
NAME      UUID                                  TYPE            DEVICE
enp8s0f0  86b340dd-08f8-4067-95da-64e5d065c9e2  802-3-ethernet  enp8s0f0
enp8s0f1  5ddd53ce-04fc-4def-8c28-142c5c1e3f4a  802-3-ethernet  --
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp8s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether a8:0c:0d:xx:xx:xxe brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.100/24 brd 192.168.0.255 scope global enp8s0f0
       valid_lft forever preferred_lft forever
    inet6 fe80::aa0c:dff:xxxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever
3: enp8s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether a8:0c:0d:xx:xx:xx brd ff:ff:ff:ff:ff:ff

본딩을 추가하는

$ sudo nmcli connection add type bond autoconnect no con-name jamesbond ifname jamesbond mode active-backup
Connection 'jamesbond' (3f972a86-feaf-4fd3-89c0-0b781d76aa18) successfully added.
$ nmcli c
NAME       UUID                                  TYPE            DEVICE
enp8s0f0   86b340dd-08f8-4067-95da-64e5d065c9e2  802-3-ethernet  enp8s0f0
jamesbond  3f972a86-feaf-4fd3-89c0-0b781d76aa18  bond            --
enp8s0f1   5ddd53ce-04fc-4def-8c28-142c5c1e3f4a  802-3-ethernet  --

노예를 추가하는 것

한마디로 사용하지 않는 인터페이스를 노예로 추가

$ sudo nmcli connection add type bond-slave autoconnect no ifname enp8s0f1 master jamesbond
Connection 'bond-slave-enp8s0f1' (30170a37-4157-44be-99dd-9e99442839ff) successfully added.
$ nmcli c
NAME                 UUID                                  TYPE            DEVICE
jamesbond            3f972a86-feaf-4fd3-89c0-0b781d76aa18  bond            --   
enp8s0f0             86b340dd-08f8-4067-95da-64e5d065c9e2  802-3-ethernet  enp8s0f0
bond-slave-enp8s0f1  30170a37-4157-44be-99dd-9e99442839ff  802-3-ethernet  --   
enp8s0f1             5ddd53ce-04fc-4def-8c28-142c5c1e3f4a  802-3-ethernet  --

연결. *을 확인하고 jamesbond의 노예 같은 것 그것이가 우리에게 있습니다

$ nmcli -f connection c s bond-slave-enp8s0f1
connection.id:                          bond-slave-enp8s0f1
connection.uuid:                        30170a37-4157-44be-99dd-9e99442839ff
connection.interface-name:              enp8s0f1
connection.type:                        802-3-ethernet
connection.autoconnect:                 no
connection.timestamp:                   0
connection.read-only:                   no
connection.permissions:
connection.zone:                        --
connection.master:                      jamesbond
connection.slave-type:                  bond
connection.secondaries:
connection.gateway-ping-timeout:        0

자동 연결이 유효 할 때 DHCP 서버가 사용하는 것과이 시점에서 IP 주소가 진동 된 경우가 있습니다.

본딩 휄로우에 IP 주소 등을 설정하는 것

설정 전

$ nmcli -f ipv4 c s jamesbond
ipv4.method:                            auto
ipv4.dns:
ipv4.dns-search:
ipv4.addresses:
ipv4.routes:
ipv4.ignore-auto-routes:                no
ipv4.ignore-auto-dns:                   no
ipv4.dhcp-client-id:                    --
ipv4.dhcp-send-hostname:                yes
ipv4.dhcp-hostname:                     --
ipv4.never-default:                     no
ipv4.may-fail:                          yes

설정

$ sudo nmcli c e jamesbond
nmcli> set ipv4.method manual
nmcli> set ipv4.dns 192.168.0.5,192.168.0.6
nmcli> set ipv4.dns-search example.com
nmcli> set ipv4.addresses 192.168.0.100/24 192.168.0.1
nmcli> p ipv4
['ipv4' setting values]
ipv4.method:                            manual
ipv4.dns:                               192.168.0.5, 192.168.0.6
ipv4.dns-search:                        example.com
ipv4.addresses:                         { ip = 192.168.0.100/24, gw = 192.168.0.1 }
ipv4.routes:
ipv4.ignore-auto-routes:                no
ipv4.ignore-auto-dns:                   no
ipv4.dhcp-client-id:                    --
ipv4.dhcp-send-hostname:                yes
ipv4.dhcp-hostname:                     --
ipv4.never-default:                     no
ipv4.may-fail:                          yes
nmcli> verify
Verify connection: OK
nmcli> save
Connection 'jamesbond' (3f972a86-feaf-4fd3-89c0-0b781d76aa18) successfully updated.
nmcli> quit

재시작에서 자동 연결을 조정할 것

sudo sed -i -e 's/ONBOOT=no/ONBOOT=yes/' /etc/sysconfig/network-scripts/ifcfg-bond-slave-enp8s0f1
sudo sed -i -e 's/ONBOOT=no/ONBOOT=yes/' /etc/sysconfig/network-scripts/ifcfg-jamesbond
sudo nmcli c m enp8s0f0 connection.autoconnect no

재시작

$ sudo service network restart; sudo nmcli c up bond-slave-enp8s0f1

1 행으로 입력하지 않아도 노예가 올라갔습니다. 그런데 네트워크 액세스가 눈에 띄지 않게 낫습니다.도, OS에서 다시 부팅 할 때까지 할 수 있습니다. 에 걸레가 없다는 것.

심지어 한쪽의 페이스 북도 본딩에 추가

$ sudo nmcli connection add type bond-slave ifname enp8s0f0 master jamesbond

이번은 autoconnect의 지정은 없습니다 (기본값 yes)로 ok

$ nmcli c
NAME                 UUID                                  TYPE            DEVICE
jamesbond            0f731c67-075e-447a-8ddb-c7a4b9dd0d81  bond            jamesbond
bond-slave-enp8s0f1  9bb37dd8-24ed-4335-9e5e-b3a1da8fa51c  802-3-ethernet  enp8s0f1
enp8s0f1             5ddd53ce-04fc-4def-8c28-142c5c1e3f4a  802-3-ethernet  --   
bond-slave-enp8s0f0  bdb5239b-a868-40e2-8ff7-81be4313efac  802-3-ethernet  enp8s0f0
enp8s0f0             86b340dd-08f8-4067-95da-64e5d065c9e2  802-3-ethernet  --
$ cat /proc/net/bonding/jamesbond
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: enp8s0f1
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: enp8s0f1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a8:0c:0d:xx:xx:xx
Slave queue ID: 0

Slave Interface: enp8s0f0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a8:0c:0d:xx:xx:xx
Slave queue ID: 0

つ な で は, 노예를 갈아 입히다

$ sudo ifenslave -c jamesbond enp8s0f0
$ grep 'Active Slave:' /proc/net/bonding/jamesbond
Currently Active Slave: enp8s0f0
$ sudo ifenslave -c jamesbond enp8s0f1
$ grep 'Active Slave:' /proc/net/bonding/jamesbond
Currently Active Slave: enp8s0f1

bonding을 해제하는

enp8s0f1이 (가) Active Slave에서 enp8s0f0을 (를) 설정하지 않도록 설정

$ nmcli c
NAME                 UUID                                  TYPE            DEVICE
jamesbond            0f731c67-075e-447a-8ddb-c7a4b9dd0d81  bond            jamesbond
bond-slave-enp8s0f1  9bb37dd8-24ed-4335-9e5e-b3a1da8fa51c  802-3-ethernet  enp8s0f1
enp8s0f1             5ddd53ce-04fc-4def-8c28-142c5c1e3f4a  802-3-ethernet  --   
bond-slave-enp8s0f0  bdb5239b-a868-40e2-8ff7-81be4313efac  802-3-ethernet  enp8s0f0
enp8s0f0             86b340dd-08f8-4067-95da-64e5d065c9e2  802-3-ethernet  --
$ sudo nmcli c del bond-slave-enp8s0f0
$ nmcli c
NAME                 UUID                                  TYPE            DEVICE
jamesbond            0f731c67-075e-447a-8ddb-c7a4b9dd0d81  bond            jamesbond
bond-slave-enp8s0f1  9bb37dd8-24ed-4335-9e5e-b3a1da8fa51c  802-3-ethernet  enp8s0f1
enp8s0f1             5ddd53ce-04fc-4def-8c28-142c5c1e3f4a  802-3-ethernet  --   
enp8s0f0             86b340dd-08f8-4067-95da-64e5d065c9e2  802-3-ethernet  --

bond-slave-enp8s0f0가 사라 졌으므로 enp8s0f0를 다시 설정하지만, 앞의 것이 남아있는 것이 자동 연결을하는 것뿐입니다.

$ nmcli -f connection,ipv4 c s enp8s0f0
connection.id:                          enp8s0f0
connection.uuid:                        86b340dd-08f8-4067-95da-64e5d065c9e2
connection.interface-name:              --
connection.type:                        802-3-ethernet
connection.autoconnect:                 no
connection.timestamp:                   1407189160
connection.read-only:                   no
connection.permissions:
connection.zone:                        --
connection.master:                      --
connection.slave-type:                  --
connection.secondaries:
connection.gateway-ping-timeout:        0
ipv4.method:                            manual
ipv4.dns:                               192.168.0.5, 192.168.0.6
ipv4.dns-search:                        example.com
ipv4.addresses:                         { ip = 192.168.0.100/24, gw = 192.168.0.1 }
ipv4.routes:
ipv4.ignore-auto-routes:                no
ipv4.ignore-auto-dns:                   no
ipv4.dhcp-client-id:                    --
ipv4.dhcp-send-hostname:                yes
ipv4.dhcp-hostname:                     --
ipv4.never-default:                     no
ipv4.may-fail:                          yes

nmcli connection edit enp8s0f0set connection.autoconnect yesし てsave과연, す ぐ に に し ちゃ う から modify コ マン ド を 使 い と.

bonding 인터 페이스가 자동으로 작동하지 않는 것처럼, enp8s0f0이 작동하는 것처럼

$ sudo nmcli c m jamesbond connection.autoconnect no
$ sudo nmcli c m bond-slave-enp8s0f1 connection.autoconnect no
$ sudo nmcli c m enp8s0f0 connection.autoconnect yes

네트워크 서비스 재시작

$ sudo service network restart

본딩을 掃除

$ sudo nmcli c del bond-slave-enp8s0f1
$ sudo nmcli c del jamesbond
$ nmcli c
NAME      UUID                                  TYPE            DEVICE
enp8s0f1  5ddd53ce-04fc-4def-8c28-142c5c1e3f4a  802-3-ethernet  --
enp8s0f0  86b340dd-08f8-4067-95da-64e5d065c9e2  802-3-ethernet  enp8s0f0

'IBM PowerLinux > REDHAT' 카테고리의 다른 글

Hot add, remove, rescan of SCSI devices on Linux  (0) 2018.07.09
Redhat Nouveau 내장그래픽카드 nvidia 드라이버 설치 삭제  (0) 2018.07.03
gnome login screen id  (0) 2016.08.11
Linux semaphore  (0) 2016.07.11
NETWORK Bonding  (0) 2016.06.29

 

nmcli 구성

 

 

CentOS7에서 새로운 커멘드라인인 nmcli 명령으로 본딩 구성 하기

1. 본딩에 사용할 디바이스 확인

[root@server] /root># nmcli dev

장치    유형      상태            연결         

eno1    ethernet  연결됨          eno1   

ens3f0  ethernet  연결됨          ens3f0 

 

2. 기존 Connection 확인 후 삭제

[root@server] /root># nmcli con

이름          UUID                                  유형            장치   

eno1    d69a2e73-63c2-424a-af09-dc67305fc45c  802-3-ethernet  eno1   

ens3f0  d524f46c-e872-46dd-9a22-acac19b07904  802-3-ethernet  ens3f0 

[root@server] /root># nmcli con del eno1

[root@server] /root># nmcli con del ens3f0

 


3. 본딩 인터페이스 추가 하기

[root@server] /root># nmcli con add type bond con-name bond0 ifname bond0 mode active-backup
Connection 'bond0' (7e9ac222-5445-484d-9aaa-81c9681980bb) successfully added.

 

4. 만들어진 본딩 인터페이스 bond0에 ip 수동 설정 (생략하면 디폴트로 dhcp가 구성 된다)

[root@server] /root># nmcli con mod bond0 ipv4.addresses 192.168.0.2/24
[root@server] /root># nmcli con mod bond0 ipv4.gateway 192.168.0.1
[root@server] /root># nmcli con mod bond0 ipv4.method manual

 

5. 슬레이브 인터페이스 추가 (1번에서 확인한 디바이스 이름 사용)

[root@server] /root># nmcli con add type bond-slave con-name bond0-eth0 ifname eno1 master bond0
Connection 'bond0-eth0' (29b2492c-222f-43fe-a8cd-da1ab35f9131) successfully added.

[root@server] /root># nmcli con add type bond-slave con-name bond0-eth1 ifname ens3f0 master bond0
Connection 'bond0-eth1' (d9fce37a-fc58-4210-b9fc-601316cb031d) successfully added.

 

6. bond0 인터페이스를 up 시킨다

[root@server] /root># nmcli con up bond0
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6) 

^^본딩 설정 완료

 

본딩 설정 후 확인 하는 방법

[root@server] /root># nmcli con

이름          UUID                                  유형            장치   

bond0         f3b60e92-e36b-4ae0-b8f1-636fb56006a8  bond            bond0  

bond0-eno1    d69a2e73-63c2-424a-af09-dc67305fc45c  802-3-ethernet  eno1   

bond0-ens3f0  d524f46c-e872-46dd-9a22-acac19b07904  802-3-ethernet  ens3f0


[root@server] /root># cat /proc/net/bonding/bond0 



출처: http://fendys.tistory.com/28 []

 

 

 

 

 

 

 

 

 

[RHEL7] NIC teaming 구성 방법 - 네트워크 이중화 (nmcli 사용)

 

- RHEL7 버전부터는 nmcli를 통하여 NIC 티밍이 가능하다. 7이하 버전에서 사용하던 방식인 bonding도 물론가능하지만 nmcli 를 통해 NIC를 이중화 구성해보자

 

1. 인터페이스 확인
# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master Team0 state UP mode DEFAULT qlen 1000
link/ether 00:50:56:9b:24:4b brd ff:ff:ff:ff:ff:ff
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master Team0 state UP mode DEFAULT qlen 1000
link/ether 00:50:56:9b:24:4b brd ff:ff:ff:ff:ff:ff
8: Team0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 00:50:56:9b:24:4b brd ff:ff:ff:ff:ff:ff

2. 팀연결 Team0 생성
# nmcli con add type team con-name Team0 ifname Team0
config '{"runner":{"name":"activebackup"}}'

# 주의~! 컬럼 및 괄호 빠질경우 디바이스 인식자체가 되지 않음



3. Team0에 대한 ipv4 설정
# nmcli con mod Team0 ipv4.addresses '165.213.246.94/24 165.213.246.1'
# nmcli con mod Team0 ipv4.method manual


4. ens192, ens224에 대한 포트 인터페이스 지정
# nmcli con add type team-slave con-name Team0-port1 ifname ens192 master Team0
# nmcli con add type team-slave con-name Team0-port2 ifname ens224 master Team0

5. 기존연결 자동 비활성화, Team-port 연결 활성화 설정
# nmcli con mod ens192 connection.autoconnect no

# nmcli con mod ens224 connection.autoconnect no
# nmcli con mod Team0-port1 connection.autoconnect yes

# nmcli con mod Team0-port2 connection.autoconnect yes

6. 연결 활성화 시작
# nmcli con up Team0

7. 서비스 재시작 및 현재 구성 확인
# systemctl restart NetworkManager
# teamdctl Team0 state

setup:
runner: activebackup
ports:
ens192
link watches:
link summary: up
instance[link_watch_0]:
name: ethtool
link: up
ens224
link watches:
link summary: up
instance[link_watch_0]:
name: ethtool
link: up
runner:
active port: ens192

8. 연결 끊기 테스트
# nmcli dev dis ens192
# teamdctl team0 state
setup:
runner: activebackup
ports:
ens224
link watches:
link summary: up
instance[link_watch_0]:
name: ethtool
link: up
runner:
active port: ens224



출처: http://yangnoon.tistory.com/10 [양눈의 IT 블로그]

 

 

 

 

 

 

 

https://www.suse.com/documentation/sles-12/stor_admin/data/sec_multipath_best_practice.html#sec_multipath_best_practice_san_timeout

 

 

 

Using Multipath with NetApp Devices

When using multipath for NetApp devices, we recommend the following settings in the /etc/multipath.conf file:

  • Set the default values for the following parameters globally for NetApp devices:

    max_fds max
    queue_without_daemon no
  • Set the default values for the following parameters for NetApp devices in the hardware table:

    dev_loss_tmo infinity
    fast_io_fail_tmo 5
    features "3 queue_if_no_path pg_init_retries 50"

 

SAN Timeout Settings When the Root Device Is Multipathed

A system with root (/) on a multipath device might stall when all paths have failed and are removed from the system because a dev_loss_tmo timeout is received from the storage subsystem (such as Fibre Channel storage arrays).

If the system device is configured with multiple paths and the multipath no_path_retry setting is active, you should modify the storage subsystem’s dev_loss_tmo setting accordingly to ensure that no devices are removed during an all-paths-down scenario. We strongly recommend that you set the dev_loss_tmo value to be equal to or higher than the no_path_retry setting from multipath.

The recommended setting for the storage subsystem’s dev_los_tmo is

<dev_loss_tmo> = <no_path_retry> * <polling_interval>

where the following definitions apply for the multipath values:

  • no_path_retry is the number of retries for multipath I/O until the path is considered to be lost, and queuing of IO is stopped.

  • polling_interval is the time in seconds between path checks.

Each of these multipath values should be set from the /etc/multipath.conf configuration file. For information, see Section 16.6, Creating or Modifying the /etc/multipath.conf File.

https://www.ibm.com/developerworks/community/blogs/meyawi/entry/dlpar_issues_with_cloned_aix_lpar?lang=en

'IBM AIX > VIOS' 카테고리의 다른 글

VIOS HOW TO  (0) 2016.09.26
vios  (0) 2016.07.19
LPM by IBM  (0) 2016.07.18
LPM  (0) 2016.07.18
VSCSI  (0) 2016.07.18



https://www.suse.com/documentation/sles-12/stor_admin/data/sec_multipath_trouble.html

'IBM PowerLinux > SUSE' 카테고리의 다른 글

SUSE 12 multipath 설정 후 부팅 안될때  (0) 2017.01.12
SAP HANA system  (0) 2016.05.20
Storage Administration Guide  (0) 2016.05.19
multipath  (0) 2016.05.19

+ Recent posts