Standby Control Panel Server

This guide describes how to prepare a standby control panel (CP) server, which can be used to recover services when the primary control panel server has an outage or requires maintenance.

 To activate the standby CP, do the following:

1. On the standby CP, perform the regular CP server installation process. Ensure that the installed package versions are the same on both control panel servers.

2. On the standby CP, create a directory to store copies of the CP configuration files and database dumps.

   [root@onapp-cp2 ~]#  mkdir /root/BACKUP_DIR/

3. From the primary CP, share the root and OnApp users’ public RSA keys with the standby CP.

   [root@onapp-cp1 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub root@[Standby CP IP address]
   [root@onapp-cp1 ~]# ssh-copy-id -i /home/onapp/.ssh/id_rsa.pub root@[Standby CP IP address]

4. Add the standby CP’s management IP address to the following line in the file /onapp/onapp-cp.conf (on the active CP).

   DB_DUMP_SERVER=""

5. Change the line:

   From	To
   DB_DUMP_SERVER_ROOT="/onapp/backups"	DB_DUMP_SERVER_ROOT="/root/BACKUP_DIR/"

   After these changes, the hourly database dumps will be transferred to the standby CP automatically.
   
   

6. On the active CP, copy the needed configuration files:

   [root@onapp-cp1 ~]# mkdir /BACKUP_DIR/
   [root@onapp-cp1 ~]# cd /BACKUP_DIR/ 
   [root@onapp-cp1 ~]# tar czf etc.tar.gz /etc/ 
   [root@onapp-cp1 ~]# tar cjf onapp_interface.tar.bz2  --exclude=/onapp/interface/log --exclude=/onapp/interface/db/dump /onapp/interface
   [root@onapp-cp1 ~]# cp -a /root/.ssh ./ssh_root 
   [root@onapp-cp1 ~]# cp -a /home/onapp/.ssh ./ssh_onapp
   [root@onapp-cp1 ~]# rsync --progress -avz -e ssh /BACKUP_DIR/ root@[Standby CP IP address]:/root/BACKUP_DIR/

7. On the active CP, add the following lines to /etc/crontab to enable automatic (hourly) backup and replication of the Redis database:

   10 * * * * root echo "SAVE" | redis-cli -s /var/run/redis/redis.sock -x
   20 * * * * root  rsync -az -e ssh /var/lib/redis/onapp.rdb root@[Standby CP IP address]:/root/BACKUP_DIR/

8. If СloudBoot is used, also add the following lines to /etc/crontab to enable automatic (daily) replication of CloudBoot configuration files:

   5 0 * * * root rsync -az -e ssh /tftpboot root@[Standby CP IP address]:/root/BACKUP_DIR/
   30 0 * * * root rsync -az -e ssh /onapp/configuration/dhcp/dhcpd.conf root@[Standby CP IP address]:/root/BACKUP_DIR/

If there is an outage on the active CP, refer to the Control Panel Migration Guide. Use steps 9 to 27 to activate the standby CP and restore services. For step 24, the RabbitMQ credentials can be found using the following command:

[root@onapp-cp2 ~]# cat /root/BACKUP_DIR/onapp/interface/config/on_app.yml|grep rabbit