Upgrade Cloud with CloudBoot Servers (from 6.7)

Upgrade CloudBoot Packages

To upgrade the CloudBoot, proceed with the steps from one to six (1-3) on Control Panel box:

1. Upgrade the repo:

   # rpm -Uvh http://rpm.repo.onapp.com/repo/onapp-repo-6.8.noarch.rpm
   

2.  Update the corresponding package(s):

   # yum update onapp-ramdisk-centos7-default onapp-ramdisk-centos7-kvm

   After the packages installation, go to your Control Panel > Admin > Settings > Configuration icon and click the Save Configuration button.

3. Run the script:

   # /onapp/onapp-store-install/onapp-store-install.sh

   Be aware that the disk-less nodes password is the root password for the CloudBoot compute resources. By default it is blank.

   
   When run in the interactive mode, enter the required information.

Upgrade CloudBoot Backup Servers

CloudBoot backup servers are CloudBoot KVM compute resources that can be used as backup servers. The CloudBoot backup server upgrade procedure is almost the same as the CloudBoot compute resource upgrade. Follow the instructions provided in this section to upgrade CloudBoot backup servers in your cloud.

Once you have upgraded the CloudBoot dependencies, you have to reboot your CloudBoot compute resource to update the Cloud Boot RPM. You do not need to perform any backup server upgrade operations using console.

To do so:

1. Go to your Control Panel > Admin > Settings menu.

2. Click the Compute Resources icon.

3. Click the label of the CloudBoot compute resource the backup server is based on.

4. On the compute resource details screen, click the Actions button, then click Reboot Compute resource.
   

5. A new screen will open asking for confirmation before reboot:
   
   
   • Are you sure you want to reboot this compute resource? Confirm that you want the compute resource to reboot.
     
     

6. When you're certain you want to proceed with the reboot, click the Reboot button.

7. Repeat these steps for all CloudBoot backup servers in your cloud.

8. Once all are rebooted, proceed to CloudBoot compute resources upgrade.

Upgrade CloudBoot Compute Resources

Depending on the infrastructure, scale and needs of your cloud we suggest the following methods of upgrading CloudBoot compute resources:

In case you have applied any custom configuration to your CloudBoot servers, it is recommended to recheck that this customization does not break new cloud boot image version. For this, reboot a compute resource and run Storage Health Check and Network Health Check. Make sure that vDisks hosted on a compute resource are redundant and healthy before rebooting a CloudBoot compute resource.

Simple Reboot

Follow the below procedure to upgrade the CloudBoot compute resources with reboot:

1. Upgrade CloudBoot Packages.
2. When the CloudBoot packages upgrade is complete, stop all virtual servers which reside on the CloudBoot compute resources.
3. Reboot all CloudBoot compute resources.

Once the compute resources are booted, the upgrade is complete. Before starting all virtual servers please ensure that the diagnostics page does not report any issue. In case of any issue, please click repair button to resolve it, then continue with starting virtual servers.

Migrate and Reboot

Use this procedure if you prefer migrating all virtual servers to another compute resource and conducting overall upgrade of your CloudBoot and Integrated Storage. Virtual servers that do not support hot migrate will have to be stopped.

Once you have upgraded the CloudBoot packages, you have to reboot your CloudBoot compute resources to update them.

To do so:

1. Run the following command from the Control Panel server terminal to display the list of compute resources with their IP addresses. Make a note of the list of IPs:

   CP_host# liveUpdate listHVs

   If the command liveUpdate is not available, then it may be located in the sbin directory instead (cd /usr/local/sbin).

2. Migrate all the virtual servers from the CloudBoot compute resource to another compute resource. Follow the instructions described in the Migrate Virtual Server section of the Admin guide to migrate virtual servers.

   Before you migrate, reboot, or start up virtual servers, please ensure that grub2.img is present on the destination compute resource in the /onapp/tools directory. To add the file to all compute resources, please run the following commands on the Control Panel server before the corresponding actions:

   cd /onapp/;wget http://templates.repo.onapp.com/Linux/grub2.img
   for i in `cat /onapp/configuration/dhcp/dhcpd.conf | grep fixed | sed 's/;//' | awk '{print $2}'`; do echo -n "$i -> "; scp /onapp/grub2.img $i:/onapp/tools/; done

3. After that, go to your Control Panel > Admin > Settings menu.

4. Click the Compute Resources icon.

5. Click the label of the CloudBoot compute resource you have migrated all VSs from.

6. On the compute resource details screen, click the Actions button, then click Reboot Compute resource.

   Rebooting a compute resource assigned to a data store with a single replica (single-replica compute resource) or degraded virtual disks may result in data loss.

7. A new screen will open asking for confirmation (via two check boxes) before reboot:
   
   
   • Stop all virtual servers that cannot be migrated to another compute resource? Check this box if you want VSs that cannot be migrated to be powered off. When a compute resource is scheduled for a reboot, OnApp will first attempt to hot migrate all VSs it hosts. If hot migration is not possible for a VS, OnApp will attempt to cold migrate that VS. With this box checked, if cold migration fails, the VS will be stopped so the reboot may proceed. If you don't check this box, OnApp will attempt to hot and then cold migrate all VSs hosted by the compute resource being rebooted – but will stop the migration process if any VS cannot be migrated.

   • Are you sure you want to reboot this compute resource? A simple confirmation to confirm that you want the compute resource to reboot.

     Before the reboot, please ensure that all vDisks are fully synced and redundant. If some of them are not fully synced, the virtual server, that is owner of a degraded (or non-redundant) vDisk, can loose access to the vDisk. It can be manifested as IO errors during writes or reads to/from the vDisk inside the virtual server.

8. When you're certain you want to proceed with the reboot, click the Reboot button.

9. Repeat these steps for all CloudBoot compute resources in your cloud.

Live Upgrade

Use this procedure to upgrade without rebooting your servers:

1. Make sure no disks are out of sync. To check the diagnostics page, go to your Control Panel > Storage > compute zone label > Diagnostics menu. Alternatively, log in to a compute resource and run the command below: 
   

   HV_host#> getdegradedvdisks

   
   

2. Repair all the degraded disks before proceeding to the upgrade process. To do so, go to your Control Panel > Storage > compute zone label > Diagnostics menu. Alternatively, run one of the following commands:

   • To repair a specific vDisk, use the following command:
     

     HV_host#> onappstore repair uuid=

   • To repair all vDisks one by one, use the following command:
     

     HV_host#>repairvdisks

   • To repair all vDisks in 10 threads simultaneously, use the following command:
     

     HV_host#> parallelrepairvdisks

     
     Please note that parallelrepairvdisks command performs the repairs much faster but impacts the Integrated Storage SAN network. The vDisk performance may be slower during the repair.
     

     In case you have a Cloudboot backup server, you can perform these commands on the backup server. The repairs will be triggered across all CloudBoot compute zones.

3. Run the following command from the CP server to stop the OnApp service:
   

   CP_host#> systemctl stop onapp

4. Stop the Apache server:
   

   CP_host#> service httpd stop

5. Make sure to update CloudBoot packages before proceeding to the following steps.

6. Run the following command from the Control Panel server terminal to display the list of compute resources with their IP addresses. Make a note of the list of IPs:
   

   CP_host#> liveUpdate listHVs 

   This command will also show whether compute resources are eligible for the live upgrade.
   

   If the command liveUpdate is unavailable, it may be located in the sbin directory instead (cd /usr/local/sbin).

   
   

7. Run the following command for every compute resource:
   

   CP_host#> liveUpdate updateToolstack <HV IP Addr> 

   Once all the toolstacks are updated, run the following command for every compute resource: 
   

   CP_host#> liveUpdate refreshControllers <HV IP Addr>

   Wait several minutes for all degraded disks to come to a synchronized state. The synchronization will take approximately three minutes for each compute resource.

   After each controller restart, check for any issues on the backup server (or on one compute resource from each zone):

   1. Log on to the backup server (or compute resource) via SSH.
   2. Run getdegradednodes from the SSH console.
   3. Run getdegradedvdisks from the SSH console.

8. Restart the storage controllers. This command can be performed later at a more suitable time.
   Run the following command for each compute resource in turn:
   

   CP_host#> liveUpdate restartControllers <HV IP Addr>

   Please make sure you restart all controllers and don’t leave your cloud in a partially updated state for too long. Note that you cannot use the disk hot plug when operating in LiveUpdate mode (for example, with the tool stacks updated but before you have performed the controller restart).

   After each controller restart, check for any issues on the backup server or one compute resource from each zone:
   1. Log on to the backup server (or compute resource) via SSH.
   2. Run getdegradednodes from the SSH console.
   3. Run getdegradedvdisks from the SSH console.
   If there are any issues seen, please rectify them before continuing with the next controller restart.

   
   

9. Run the following command on each compute resource to make sure the package versions are upgraded:
   

   HV_host#> cat /onapp/onapp-store-install.version

10. Start the Apache server:
    

    CP_host#> service httpd start

11. Start the OnApp service:
    

    CP_host#> service onapp start