CentOS6 to CentOS7 Control Panel Migration Guide

This Control Panel Migration Guide describes the migration procedure both for clouds with Static Hypervisors and for clouds with CloudBoot Hypervisors.

According to recommendations from MariaDB, migration from MySQL database to MariaDB database should be carried out through mysql_upgrade in order to eliminate the following problems:

  • You will get warnings in the error log every time you access a table with an invalid (old) file name
  • When searching on key values you may not find all rows
  • You will get an error "ERROR 1459 (HY000): Table upgrade required" when accessing the table.
  • You may get crashes

In this case, you can migrate the CentOS6 server's database to MariaDB using the specific script and then restore MariaDB backup on a new server.

Ensure that the failover is disabled on the source CP server before starting cloud data migration. You can enable failover on the new CP server after all migration tasks are finished, and CR is manageable by the new CP.

Take the following steps to perform migration:

  1. Install OnApp packages on the new CP server. Package versions of both CPs must be the same. It should be done according to Install Control Panel Server. The only difference is in step 5, where MariaDB must be installed with this command:

    /onapp/onapp-cp-install/onapp-cp-install.sh -i SNMP_TRAP_IPS 
  2. Stop OnApp and httpd daemons on the old CP:

    service onapp stop
    service httpd stop
    service onapp-licensing stop
    service crond stop
    service monit stop 
  3. Ensure OnApp and httpd daemons are stopped

    ps ax |grep onappd  
    ps ax |grep httpd
  4. Make dump of OnApp database; /etc/ folder; .ssh/ folder (for root and OnApp users); /onapp/interface/ :

    mkdir /BACKUP_DIR/
    cd /BACKUP_DIR/ 
    cp -a  /onapp/configuration ./
    grep pass /onapp/interface/config/database.yml 
    mysqldump --extended-insert onapp > onapp.sql
    tar czf etc.tar.gz /etc/ 
    tar cjf onapp_interface.tar.bz2  --exclude=/onapp/interface/log --exclude=/onapp/interface/db/dump /onapp/interface
    cp -a /root/.ssh ./ssh_root 
    cp -a /home/onapp/.ssh ./ssh_onapp
  5. Versions 3.3.0 and later require backing up of Redis database.

    echo "SAVE" | redis-cli -s /var/run/redis/redis.sock -x
    ll /var/lib/redis/onapp.rdb
    cp -a /var/lib/redis/onapp.rdb . 

    Be aware, that redis.sock can be located in tmp/ directory.

  6. Disable OnApp and httpd daemons in autorun:

    For CentOS6:

    chkconfig onapp off 
    chkconfig httpd off
    chkconfig onapp-licensing off

    For CentOS7:

    chkconfig onapp off 
    systemctl disable httpd.service
    chkconfig onapp-licensing off
  7. Ensure those daemons were really disabled:

    chkconfig --list |egrep "onapp|httpd"
  8. Copy all dumped data to new CP server (create /root/BACKUP_DIR on remote host):

    rsync --progress -avz -e ssh /BACKUP_DIR/ user@host:/root/BACKUP_DIR/
  9. Drop the new database on the new Control Panel server:

    cd /root/BACKUP_DIR/
    mysql -p onapp 
    drop database onapp;
    show databases;
    create database onapp; 
    show databases;
  10. Apply database from the old CP to the new one:

    mysql onapp < onapp.sql 
  11. Apply Redis database if CP's version is 3.3.0 or later

    Make sure the Redis database file is the /var/lib/redis/onapp.rdb file as it is assumed that the dbfilename onapp.rdb and dir var/lib/redis directives are set in /etc/redis.conf.

    service redis stop
    mv /var/lib/redis/onapp.rdb onapp.rdb.old
    cp -a onapp.rdb /var/lib/redis/
    service redis start
    service redis status
  12. Apply SSH files for root and onapp users:

    rm /root/.ssh/*
    chmod 700 /root/.ssh/
    rm /home/onapp/.ssh/*
    cp -a ssh_root/* /root/.ssh/
    cp -a ssh_onapp/* /home/onapp/.ssh/
  13. Apply /onapp/interface/config:

    mv /onapp/interface/config /onapp/interface/config.old
    yum install bzip2
    tar jxf onapp_interface.tar.bz2
    cp -a ./onapp/interface/config /onapp/interface/
    cp /onapp/interface/config.old/database.yml /onapp/interface/config/database.yml
  14. Check the owner of ssh keys in /onapp/interface/config/keys (in layout owner and group must be onapp):

    ll /onapp/interface/config/keys
    -rw------- 1 onapp onapp 1675 Dec 19 08:39 private
    -rw-r--r-- 1 onapp onapp 382 Dec 19 08:39 public
  15. Apply exports file to new CP:

    tar zxf etc.tar.gz
    cp ./etc/exports /etc/

    Perform the following steps 16-20 only if your Cloud uses CloudBootHypervisors. For the cloud with StaticHypervisors - proceed to step 21

  16. Copy TFTPBoot folder from the old CP to the new one:

    rsync --progress -avz -e ssh /tftpboot user@host:/copied_backups_from_old_cp/
  17. Copy /onapp/configuration/dhcp/dhcpd.conf from the old CP to the new one:

    rsync --progress -avz -e ssh /onapp/configuration/dhcp/dhcpd.conf user@host:/copied_backups_from_old_cp/

    For OnApp versions previous to 4.0 file location is /home/onapp/dhcpd.conf

  18. Stop dhcpd daemon on the old CP and disable it in the autorun:

    chkconfig dhcpd off
    service dhcpd stop
    ps ax |grep dhcpd
    service redis stop
    chkconfig redis off
    chkconfig --list |egrep "onapp|httpd|dhcpd|redis"
  19. Apply copied dhcp files and TFTPBoot folder to the new CP:

    cp etc/dhcp/dhcpd.conf /etc/dhcp/
    mkdir /onapp/configuration/dhcp/
    chown onapp:onapp /onapp/configuration/dhcp
    cp /copied_backups_from_old_cp/dhcpd.conf /onapp/configuration/dhcp/dhcpd.conf
    mv /tftpboot /tftpboot.old
    cp -a /copied_backups_from_old_cp/tftpboot /

    For OnApp versions previous to 4.0 apply the following:

    cp etc/dhcp/dhcpd.conf /etc/dhcp/
    cp /copied_backups_from_old_cp/dhcpd.conf /home/onapp/
    mv /tftpboot /tftpboot.old
    cp -a /copied_backups_from_old_cp/tftpboot /
  20. Check server_args directive in TFTP configuration. It should be server_args = -v -v -s /tftpboot:

    cat /etc/xinetd.d/tftp
  21. Make interchange of IP-addresses between old and new CP servers
  22. On the new CP - try to obtain access to HVs via SSH under the OnApp user
  23. Add onapp and httpd daemons on CP to autorun:

    chkconfig onapp on
    chkconfig onapp-licensing on
    systemctl enable httpd
    chkconfig --list | grep "onapp"
    systemctl is-enabled httpd
  24. Run the following command to apply SNMP Trap IP_address of management network and RabbitMQ the new CP:

    Perform this step if you are running OnApp 3.3.0 and above.

    For a successful migration, you need to specify the RabbitMQ credentials. You can view your RabbitMQ credentials by running the following command:

    # cat /onapp/onapp-rabbitmq/credentials.yml

    or copy /onapp/onapp-rabbitmq/credentials.yml from the RabbitMQ server standalone box, into Control Panel's /onapp/configuration/rabbit_mq/vcloud/credentials.yml. For more information, refer to RabbitMQ Server Installation.

    /onapp/onapp-cp-install/onapp-cp-install.sh [-v ONAPP_VERSION] [-i SNMP_TRAP_IPS] [--rbtlogin RBT_LOGIN] [--rbtpasswd RBT_PASSWD]
    • --rbtlogin  RBT_LOGIN - RabbitMQ manager login. The default value is 'rbtmgr'.
    • --rbtpasswd RBT_PASSWD - RabbitMQ manager password. A random password is generated if this parameter isn't specified.

    Make sure that the rbtlogin and rbtpasswd values are the same as at the Settings > Configuration page.
    Go to Settings > Configuration in the OnApp UI and click Save to complete the process.

  25. If Acceleration is enabled:

    1. Specify user name and password for rabbitmq-server:

      rabbitmqctl add_user username 'userpass'
    2. Set permissions for this user:

      rabbitmqctl set_permissions -p '/' username ".*" ".*" ".*"
    3. Restart OnApp service:

      service onapp restart

      Compute Resources and Control Panel must use the same rabbitmq-server.

    4. Make sure RabbitMQ credentials are up to date for all HVs. To do this, perform the steps, marked as "Perform the following steps if you plan to deploy Accelerator. Otherwise skip." described in the Install Compute Resources section.

  26. Run DB dump to be sure that all DB dumps are copied directly to the backup server:

    sh /onapp/onapp-cp-install/onapp-database-dump.sh
  27. After interchanging IP-addresses between the CP servers, stop all VSs running on CloudBoot hypervisors and reboot these HVs.