Configuring Control Panel Environment for Simulation Purposes (6.3 and up)

To upgrade your cloud to 6.3 Stable, you first need to migrate your databases to Maria DB 10.4. Since this process causes downtime, the duration of which depends on the cloud characteristics, you can simulate the upgrade process before the actual upgrade to be prepared for the possible downsides.

Environment preparation

To prepare the environment for the upgrade simulation, сreate a virtual server with resources as close as possible to the current actual Control Panel (RAM / CPU / storage). You can also use a baremetal server; using a VS is more flexible but some decrease in performance may occur. The example in this guide describes a case with a CentOS 7 virtual server built on the cloud where the CP is to be cloned. The same procedure will work for CentOS 6 as well.

You can calculate the approximate size of storage required for the target VS as a sum of values onapp_db_size and root_part_usage:

mysql > SELECT table_schema "DB Name", ROUND(SUM(data_length + index_length) / 1024 / 1024 / 1024, 1) "DB Size in GB" FROM information_schema.tables WHERE table_schema = "onapp";
cp# du -sh --exclude={proc,sys,dev,/onapp/backups,/onapp/templates,/data,/onapp/interface/db/dump,/onapp/interface/logs,/boot,/tmp,/var/lib/mysql,/root} /

There are the following obligatory prerequisites for the simulation:

The VS should be running the same OS version.
The VS should have internet access. Public IP is not required, but the VS should be accessible from the CP.
The management network of the production cloud should be isolated from this VS.

The simulation can be performed without a working license key and without full UI functionality. Many failed actions may occur because of no access to the compute resources and other instances. If you need full UI functionality for further extended tests of the new environment, you will need the test license. Ask your account manager to provide a test license at amteam@onapp.com.

There are two options for performing simulation:

Cloning existing Control Panel
Installing OnApp packages on the new VS and applying configuration from the backup

For the detailed instructions, refer to the corresponding section below.

Clone existing Control Panel (method 1)

To get an environment identical to the original CP, clone the existing CP system using the following instruction:

Install rsync on the VS:
```
vm# yum install rsync -y
```

Run the following step on the original CP:

cp# mkdir /BACKUP_DIR/
cp# grep pass /onapp/interface/config/database.yml
cp# mysqldump --extended-insert --password onapp | gzip - > /BACKUP_DIR/onapp.sql.gz
cp# rsync -aHXPx --delete -e "ssh -i /onapp/interface/config/keys/private -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/root/*,/boot/*,/onapp/interface/log/*,/onapp/interface/db/dump/*,/onapp/backups/*,/onapp/templates/*,/data/*,/etc/sysconfig/network-scripts/*,/etc/fstab,/etc/hostname,/lost+found} / user@host:/

Some directories are excluded as are not required but may consume a lot of space. You may append your own list.

Run the following steps on the VS box:
1. Check if the value inno_db_buffer_pull_size in the /etc/my.cnf.d/server.cnf (or/etc/my.cnf) file is optimized in comparison to the amount of the RAM allocated to the VS. It should not be higher than total RAM, <=50% is acceptable.
2. Change the license key:
```
vm# sed -i 's/^license_key.*/license_key\: NEW_KEY/g' /onapp/interface/config/on_app.yml
```
  If you don't have a test license, do the following:
```
vm# echo -e '127.0.0.1 licensing.onapp.com\n 127.0.0.1 dashboard.onapp.com' >> /etc/hosts
```
3. Do the following step if you use the CDN component:
```
vm# echo '127.0.0.1 api.onappcdn.com' >> /etc/hosts
```
4. Configure MySQL and restore the OnApp database:
```
vm# mysql -V
```
  If it’s MariaDB server:
```
vm# mkdir -p /var/run/mariadb
vm# chown mysql:mysql /var/run/mariadb 
vm# systemctl start mariadb
```
  In case of falling, check the cause of failure in the log and resolve it. It is not possible to move forward unless the database is correctly started.
5. Set the password for the user root (you may take it from /onapp/interface/config/database.yml):
```
vm# mysqladmin --user=root password "PASSWORD" 
```
6. Create the OnApp database and restore the OnApp data from the dump:
```
vm# mysql -p -e 'create database onapp;' 
vm# zcat /BACKUP_DIR/onapp.sql.gz | mysql -p onapp
```
7. Verify the boot configuration and network settings. Then reboot the VS and check that the OnApp services are running.
8. For CentOS 7 only: if OnApp service fails to start because of the dependent service onapp-ssh-agent.service failing, do the following:
```
vm#  mv /lib/systemd/system/onapp-ssh-agent.service /root/
vm# systemctl daemon-reload
```
9. Start OnApp service again:
```
vm# systemctl start onapp
```

Installing OnApp packages (method 2)

You may use a longer way of installing standard OnApp packages on the newly created VS and then applying configuration from the backup. To perform the simulation this way, do the following steps:

Update the OS components:
```
vm# yum update -y
```
Install OnApp packages on the VS. Package versions of both CPs must be the same.
```
vm#/onapp/onapp-cp-install/onapp-cp-install.sh [-v ONAPP_VERSION]
```
Check the MySQL version. If a custom database is used on the original CP then install the same on the VS. To check the MySQL version, enter the following:
```
vm# mysql --password -e 'SELECT version();'
```
or
```
vm# mysql -V
```
You may use the same /etc/my.cnf.d/server.cnf (or /etc/my.cnf) as you have on the original CP or tune it for better performance.

If you use the CDN component, do the following:

vm# echo '127.0.0.1 api.onappcdn.com' >> /etc/hosts

Create a backup of the DB and OnApp configuration files and move them to the simulated VS:

Make dump of OnApp database; /etc/; and /onapp/interface/ directories

cp# mkdir /BACKUP_DIR/
cp# cd /BACKUP_DIR/
cp# grep pass /onapp/interface/config/database.yml
cp# mysqldump --extended-insert --password onapp | gzip - > onapp.sql.gz
cp# tar cjf onapp_interface.tar.bz2  --exclude=/onapp/interface/log --exclude=/onapp/interface/db/dump /onapp/interface 
cp# tar czf etc.tar.gz /etc/

Copy all dumped data to the VS (create /root/BACKUP_DIR on the remote host):
```
cp# scp onapp.sql onapp_interface.tar.bz2 etc.tar.gz user@host:/root/BACKUP_DIR/
```
Moving original cloudboot related files (/tftpboot/*, /onapp/configuration/dhcp/dhcpd.conf) and others is not mandatory.

Restore DB and OnApp files (the VS box):

Stop OnApp related services (onapp, httpd, crond).

Apply /onapp/interface/config:

vm# mv /onapp/interface/config /onapp/interface/config.old
vm# cd /root/BACKUP_DIR/
vm# tar jxf onapp_interface.tar.bz2
vm# cp -a ./onapp/interface/config /onapp/interface/
vm# cp /onapp/interface/config.old/database.yml /onapp/interface/config/database.yml

Change CP license key:

vm#sed -i 's/^license_key.*/license_key\: NEW_KEY/g' /onapp/interface/config/on_app.yml

If you don't have a test license, do the following:

vm# echo -e '127.0.0.1 licensing.onapp.com\n 127.0.0.1 dashboard.onapp.com' >> /etc/hosts

Drop initial database on the simulated Control Panel server:

vm# mysql -p onapp 
mysql > drop database onapp;
mysql > create database onapp;

Restore the OnApp database:

vm# zcat /root/BACKUP_DIR/onapp.sql.gz | mysql -p onapp

Start OnApp related services (onapp, httpd, crond) to make sure it works.
Take a VS backup via UI to have a snapshot of the current VM’s state in case something may go wrong and perform a quick rollback.

Now you can start the simulation of the upgrade procedures: