Solved: Veritas Cluster group is not starting on two nodes as expected.
I. Issue: Veritas group is not starting on two nodes as expected by application support teams.
ngeuser1@ngeaix027> hagrp -online NGEL-threadpool_cache_now -any VCS NOTICE V-16-1-50735 Attempting to online group on system ngeaix027
II. Expectation: It should be started on both nodes.
eaixserv028 --> hagrp -online NGEL-threadpool_cache_now -any VCS NOTICE V-16-1-50735 Attempting to online group on system ngeaix027 VCS NOTICE V-16-1-50735 Attempting to online group on system ngeaix028
III. Reason: NGEL-threadpool_cache_now group needs Parallel attribute set to 1
IV. Solution/Reconfiguration:
We need to do below actions to resolve this issue.
To change Parallel attribute to “1” group needs to have no resources and no dependencies so we need to:
1) prepare commands for recreation of resources and dependencies
2) remove resources,
3) remove dependencies,
4) change Parallel = 1
5) recreate resources
6) recreate dependencies
============== Steps for reconfiguration ============ 1) Get the current configuration eaixserv028 --> hacf -cftocmd /etc/VRTSvcs/conf/config eaixserv028 --> cd /etc/VRTSvcs/conf/config eaixserv028 --> grep -E "threadpool_cache_now" main.cmd 2) Delete the Resource from SG eaixserv028 --> haconf -dump -makero eaixserv028 --> hares -delete threadpool_cache_now 3) Unlink the resources from SG eaixserv028 --> hagrp -modify NGEL-threadpool_cache_now Parallel 1 VCS WARNING V-16-1-50150 Cannot modify Parallel attribute if group dependencies have been configured for group eaixserv028 --> hagrp -unlink NGEL-threadpool_cache_now interfaces_mount hagrp -unlink NGEL-threadpool_cache_now projects_mount hagrp -unlink NGEL-threadpool_cache_now share_mount hagrp -unlink NGEL-threadpool_cache_now splebase_mount hagrp -unlink NGEL-threadpool_worker_hav NGEL-threadpool_cache_now 4) Modify the group to set parallel. eaixserv028 --> hagrp -modify NGEL-threadpool_cache_now Parallel 1 5) Add the resource back and add the configuration. eaixserv028 --> hares -add threadpool_cache_now Application NGEL-threadpool_cache_now VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors hares -modify threadpool_cache_now StartProgram "/USR/batch/batch_jobs/ngel_application.sh start threadpool_cache " hares -modify threadpool_cache_now StopProgram "/USR/batch/batch_jobs/ngel_application.sh stop threadpool_cache " hares -modify threadpool_cache_now CleanProgram "/USR/batch/batch_jobs/ngel_application.sh clean threadpool_cache " hares -modify threadpool_cache_now MonitorProgram "/USR/batch/batch_jobs/ngel_application.sh monitor threadpool_cache " hares -override threadpool_cache_now OfflineWaitLimit hares -modify threadpool_cache_now OfflineWaitLimit 8 hares -override threadpool_cache_now OnlineWaitLimit hares -modify threadpool_cache_now OnlineWaitLimit 10 hares -override threadpool_cache_now OnlineTimeout hares -modify threadpool_cache_now OnlineTimeout 600 hares -modify threadpool_cache_now User root hares -modify threadpool_cache_now PidFiles -delete -keys hares -modify threadpool_cache_now MonitorProcengels -delete -keys hares -modify threadpool_cache_now UseSUDash 0 hares -modify threadpool_cache_now Enabled 1 6) Link the SG back. hagrp -link NGEL-threadpool_cache_now interfaces_mount online local firm hagrp -link NGEL-threadpool_cache_now projects_mount online local firm hagrp -link NGEL-threadpool_cache_now share_mount online local firm hagrp -link NGEL-threadpool_cache_now splebase_mount online local firm hagrp -link NGEL-threadpool_worker_hav NGEL-threadpool_cache_now online local soft haconf –dump -makero eaixserv028 --> hacf -cftocmd /etc/VRTSvcs/conf/config eaixserv028 --> cd /etc/VRTSvcs/conf/config eaixserv028 --> grep -E "threadpool_cache_now" main.cmd