memory doesn't recognize if memory is added at the slot

이슈 상황

memory를 채널/순서에 맞게 32GB (8Gb X 4) 꽂았음에도 불구하고 16GB만 인식되는 현상

매번 구성해준 서버 혹은 세트가 거의 정확하게 맞춰져 있는 환경에서 문제없이 사용하다가
테스트를 위해 이장비 저장비에 있던 부품을 모아 만들다 보니 무언가 문제가 있는것 같았다.

환경

HP DL360 G6 장비였고 2 cpu socket에 memory는 8GB X 4개를 연결하는 과정이었다.

확인 방법

16GB만 인식된 상황에서 우선 OS 설치를 진행하였고 cpu socket에 대한 의심이 들어 아래와 같은 명령어를 통한 확인과정을 진행하였다.

실제 memory slot의 연결상태를 확인해보니 2개의 memory만 연결된 것으로 확인되었다.

[root@localhost ~]# lshw -class memory
  *-firmware                
       description: BIOS
       vendor: HP
       physical id: 0
       version: P64
       date: 03/30/2010
       size: 64KiB
       capacity: 8MiB
       capabilities: pci pnp upgrade shadowing escd cdboot bootselect edd int13floppy360 int13floppy1200 int13floppy720 int5printscreen int9keyboard int14serial int17printer int10video acpi usb biosbootspecification netboot
  *-cache:0
       description: L1 cache
       physical id: 710
       slot: Processor 1 Internal L1 Cache
       size: 128KiB
       capacity: 192KiB
       capabilities: burst internal write-back data
       configuration: level=1
  *-cache:1
       description: L2 cache
       physical id: 720
       slot: Processor 1 Internal L2 Cache
       size: 1MiB
       capacity: 1536KiB
       capabilities: burst internal write-back
       configuration: level=2
  *-cache:2
       description: L3 cache
       physical id: 730
       slot: Processor 1 Internal L3 Cache
       size: 4MiB
       capacity: 12MiB
       capabilities: burst internal write-back
       configuration: level=3
  *-memory
       description: System Memory
       physical id: 1000
       slot: System board or motherboard
       size: 32GiB
       capabilities: ecc
       configuration: errordetection=ecc
     *-bank:0
          description: DIMM DDR3 Synchronous [empty]
          physical id: 0
          slot: PROC 1 DIMM 1
          width: 64 bits
     *-bank:1
          description: DIMM DDR3 Synchronous [empty]
          physical id: 1
          slot: PROC 1 DIMM 2
          width: 64 bits
     *-bank:2
          description: DIMM DDR3 Synchronous 1333 MHz (0.8 ns)
          physical id: 2
          slot: PROC 1 DIMM 3
          size: 8GiB
          width: 64 bits
          clock: 1333MHz (0.8ns)
     *-bank:3
          description: DIMM DDR3 Synchronous [empty]
          physical id: 3
          slot: PROC 1 DIMM 4
          width: 64 bits
     *-bank:4
          description: DIMM DDR3 Synchronous [empty]
          physical id: 4
          slot: PROC 1 DIMM 5
          width: 64 bits
     *-bank:5
          description: DIMM DDR3 Synchronous 1333 MHz (0.8 ns)
          physical id: 5
          slot: PROC 1 DIMM 6
          size: 8GiB
          width: 64 bits
          clock: 1333MHz (0.8ns)
     *-bank:6
          description: DIMM DDR3 Synchronous [empty]
          physical id: 6
          slot: PROC 1 DIMM 7
          width: 64 bits
     *-bank:7
          description: DIMM DDR3 Synchronous [empty]
          physical id: 7
          slot: PROC 1 DIMM 8
          width: 64 bits
     *-bank:8
          description: DIMM DDR3 Synchronous [empty]
          physical id: 8
          slot: PROC 1 DIMM 9
          width: 64 bits
     *-bank:9
          description: DIMM DDR3 Synchronous [empty]
          physical id: 9
          slot: PROC 2 DIMM 1
          width: 64 bits
     *-bank:10
          description: DIMM DDR3 Synchronous [empty]
          physical id: a
          slot: PROC 2 DIMM 2
          width: 64 bits
     *-bank:11
          description: DIMM DDR3 Synchronous [empty]
          physical id: b
          slot: PROC 2 DIMM 3
          width: 64 bits
     *-bank:12
          description: DIMM DDR3 Synchronous [empty]
          physical id: c
          slot: PROC 2 DIMM 4
          width: 64 bits
     *-bank:13
          description: DIMM DDR3 Synchronous [empty]
          physical id: d
          slot: PROC 2 DIMM 5
          width: 64 bits
     *-bank:14
          description: DIMM DDR3 Synchronous [empty]
          physical id: e
          slot: PROC 2 DIMM 6
          width: 64 bits
     *-bank:15
          description: DIMM DDR3 Synchronous [empty]
          physical id: f
          slot: PROC 2 DIMM 7
          width: 64 bits
     *-bank:16
          description: DIMM DDR3 Synchronous [empty]
          physical id: 10
          slot: PROC 2 DIMM 8
          width: 64 bits
     *-bank:17
          description: DIMM DDR3 Synchronous [empty]
          physical id: 11
          slot: PROC 2 DIMM 9
          width: 64 bits

numactl을 통해 확인해보니 2 socket이라 2 nodes가 검색될것으로 생각했지만 1 nodes만 확인되었다.

[root@localhost ~]# numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3
node 0 size: 31998 MB
node 0 free: 29948 MB
node distances:
node   0 
  0:  10 

실제 processor를 확인해보니 cpu socket 1번이 disable 상태였다.

[root@localhost ~]# lshw -class processor
  *-cpu:0                   
       description: CPU
       product: Intel(R) Xeon(R) CPU           E5504  @ 2.00GHz
       vendor: Intel Corp.
       vendor_id: GenuineIntel
       physical id: 400
       bus info: cpu@0
       version: Intel(R) Xeon(R) CPU E5504 @ 2.00GHz
       slot: Proc 1
       size: 1600MHz
       capacity: 2GHz
       width: 64 bits
       clock: 133MHz
       capabilities: lm fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp x86-64 constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid dtherm spec_ctrl intel_stibp flush_l1d cpufreq
       configuration: cores=4 enabledcores=4 threads=4
  *-cpu:1 DISABLED
       description: CPU [empty]
       product: Xeon
       vendor: Intel
       physical id: 406
       slot: Proc 2
       capabilities: lm

이로인해 해당 socket에 연결된 memory slot은 인식자체가 안되었던 것이었다.

해결방안

CPU socket에 새로운 CPU를 꽂기는 어려워 enable 되어 있는 socket의 memory slot에 memory를 다시 연결하였다.
이렇게 하여 memory가 32GB 인식됨을 확인하였다.