error when tripleo overcloud deploying

TripleO를 통한 overcloud deploy시 에러현상

2017-03-21 08:45:07 [ControllerOvercloudServicesDeployment_Step6]: UPDATE_IN_PROGRESS state changed

2017-03-21 08:45:07 [overcloud-ControllerNodesPostDeployment-cdoooh2h6xpk-ControllerOvercloudServicesDeployment_Step6-gikl7tsipgjw]: UPDATE_IN_PROGRESS Stack UPDATE started

2017-03-21 08:45:07 [2]: UPDATE_IN_PROGRESS state changed

2017-03-21 08:45:08 [1]: UPDATE_IN_PROGRESS state changed

2017-03-21 08:45:08 [0]: UPDATE_IN_PROGRESS state changed

2017-03-21 08:46:05 [2]: SIGNAL_IN_PROGRESS Signal: deployment succeeded

2017-03-21 08:46:06 [2]: UPDATE_COMPLETE state changed

2017-03-21 08:46:30 [1]: SIGNAL_IN_PROGRESS Signal: deployment succeeded

2017-03-21 08:46:30 [1]: UPDATE_COMPLETE state changed

2017-03-21 08:46:41 [0]: SIGNAL_IN_PROGRESS Signal: deployment succeeded

2017-03-21 08:46:42 [0]: UPDATE_COMPLETE state changed

2017-03-21 08:46:43 [ControllerOvercloudServicesDeployment_Step6]: UPDATE_COMPLETE state changed

2017-03-21 08:46:43 [overcloud-ControllerNodesPostDeployment-cdoooh2h6xpk-ControllerOvercloudServicesDeployment_Step6-gikl7tsipgjw]: UPDATE_COMPLETE Stack UPDATE completed successfully

2017-03-21 08:46:44 [ControllerPostPuppet]: UPDATE_IN_PROGRESS state changed

2017-03-21 08:46:44 [overcloud-ControllerNodesPostDeployment-cdoooh2h6xpk-ControllerPostPuppet-fvjqh6eqmany]: UPDATE_IN_PROGRESS Stack UPDATE started

2017-03-21 08:46:46 [ControllerPostPuppetMaintenanceModeDeployment]: UPDATE_IN_PROGRESS state changed

2017-03-21 08:47:32 [ControllerPostPuppetMaintenanceModeDeployment]: UPDATE_COMPLETE state changed

2017-03-21 08:47:32 [ControllerPostPuppetRestartDeployment]: UPDATE_IN_PROGRESS state changed

2017-03-21 09:18:12 [ControllerPostPuppetRestartDeployment]: UPDATE_FAILED resources.ControllerPostPuppetRestartDeployment: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1

2017-03-21 09:18:13 [overcloud-ControllerNodesPostDeployment-cdoooh2h6xpk-ControllerPostPuppet-fvjqh6eqmany]: UPDATE_FAILED resources.ControllerPostPuppetRestartDeployment: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1

2017-03-21 09:18:15 [ControllerPostPuppet]: UPDATE_FAILED resources.ControllerPostPuppet: resources.ControllerPostPuppetRestartDeployment: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1

2017-03-21 09:18:16 [overcloud-ControllerNodesPostDeployment-cdoooh2h6xpk]: UPDATE_FAILED resources.ControllerPostPuppet: resources.ControllerPostPuppetRestartDeployment: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1

Stack overcloud UPDATE_FAILED

Deployment failed:  Heat Stack update failed.

위 로그와 같이 overcloud deploy 시 에러가 발생되는 경우가 있었다. 

당시 원인은 정확하지 않지만 puppet을 통한 pacemaker의 제어가 문제가 있었던것으로 보였고 

https://bugzilla.redhat.com/show_bug.cgi?id=1383780

에서 언급된 것처럼 timeout의 문제의 소지가 있어보여 아래와 같은 설정을 추가해보았다.

pcs resource op defaults timeout=60s

이후 정상적으로 배포되었다. 

실제 아래의 pacemaker_resource_restart.sh 스크립트가 수행되면서 발생되는 문제로 보여졌으며 

원인은 좀더 확인해보아야할것같다.