Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This guide explains how to upgrade OnApp Cloud v4.0 to the v4.1 for the cloud configuration where all servers are CloudBooted except Control Panel server. Please follow the complete procedure of the upgrade process. All packages (Control Panel, CloudBoot, Compute resources) must belong to the same major version to ensure the best performance of your cloud.

Note

You must be running OnApp 4.0 to upgrade to 4.1. If you are using an earlier version, please upgrade to 4.0. first.

On this page:

Table of Contents

 

 

Introduction and Important Notes


 

Excerpt Include
Introduction and Important Notes.
Introduction and Important Notes.
nopaneltrue

 

Upgrade CloudBoot Packages


Anchor
packages
packages

To upgrade the OnApp Storage packages:

  1. Upgrade the repo:

    Code Block
    bash#> rpm -Uvh http://rpm.repo.onapp.com/repo/onapp-repo-4.1.noarch.rpm
    
  2. Upgrade the packages:

    Code Block
    bash#> yum update onapp-store-install
  3. Run the script:

    Code Block
    bash#> /onapp/onapp-store-install/onapp-store-install.sh

When run in the interactive mode, enter the required information.

 

Upgrade CloudBoot Backup Servers


Info

Make sure to update CloudBoot packages before proceeding to the upgrade of CloudBoot backup servers.

CloudBoot backup servers are CloudBooted KVM compute resources that can be be used as backup servers. The CloudBoot backup server upgrade procedure is almost the same as the CloudBoot compute resource upgrade. Follow the instructions provided in this section to upgrade CloudBoot backup servers in your cloud.

Once you have upgraded the CloudBoot dependencies, you have to reboot your Cloud Boot compute resource to update the Cloud Boot RPM. You do not need to perform any backup server upgrade operations using console.

To do so:

  1. Go to your Control Panel Settings menu.

  2. Click the Compute resources icon.

  3. Click the label of the CloudBoot compute resource the backup server is based on.

  4. On the compute resource details screen, click the Actions button, then click Reboot Compute resource.

  5. A new screen will open asking for confirmation before reboot:

    • Are you sure you want to reboot this compute resource? Confirm that you want the compute resource to reboot.

  6. When you're certain you want to proceed with the reboot, click the Reboot button.

  7. Repeat these steps for all CloudBoot backup servers in your cloud.

  8. Once all are rebooted, proceed to CloudBoot compute resources upgrade.

 

Upgrade CloudBoot Compute Resources


 

Simple RebootThis method is the simplest method technically. It also ensures all tools are updated. However, it will result in some limited downtime (its duration depends on how many virtual servers are running on each compute resource).
Migrate and rebootThis method involves migrating all virtual servers off each CloudBoot compute resource in turn. The compute resource can then be safely rebooted, picking up the upgraded Integrated Storage and CloudBoot packages. Virtual servers that do not support hot migrate will have to be stopped.
Live UpgradeThis method will upgrade Integrated Storage components but will not upgrade CloudBoot image.

In case you have applied any custom configuration to your CloudBoot servers, it is recommended to recheck that this customization does not break new cloud boot image version. For this, reboot a compute resource and run Storage Health Check and Network Health Check. Make sure that Vdisks hosted on a compute resource are redundant and healthy before rebooting a CloudBoot compute resource.

Anchor
simple
simple

Simple Reboot

Follow the below procedure to upgrade the CloudBoot compute resources with reboot:
 

1. Upgrade CloudBoot Packages.
2. When the CloudBoot packages upgrade is complete, stop all virtual servers which reside on the CloudBoot compute resources.

3. Reboot all CloudBoot compute resources.

Once the compute resources are booted, the upgrade is complete. Before starting all Virtual Servers please ensure that the diagnostics page does not report any issue. In case of any issue, please press repair button to resolve it, then continue with starting Virtual Servers.

 

Info

Note that virtual servers cannot be stopped simultaneously, but must be stopped in sequence. This can result in considerable downtime if there are a large number of virtual servers.

Anchor
migrate
migrate

Migrate and reboot

Use this procedure if you prefer migrating all virtual servers to another compute resource and conducting overall upgrade of your CloudBoot and Integrated Storage. Virtual servers that do not support hot migrate will have to be stopped.

Once you have upgraded the CloudBoot packages, you have to reboot your CloudBoot compute resources to update them.

To do so:

  1.  Run the following commands from the Control Panel server terminal for each compute resource:

    Code Block
    liveUpdate updateToolstack <HV IP Addr>

    The synchronization will take approximately three minutes for each compute resource.

  2. Migrate all the virtual servers from the CloudBoot compute resource to another compute resource. Follow the instructions described in the Migrate Virtual Server section of the Admin guide to migrate virtual servers.

  3. After that, go to your Control Panel Settings menu.

  4. Click the Appliances icon.

  5. Click the label of the CloudBoot compute resource you have migrated all VSs from.

  6. On the compute resource details screen, click the Actions button, then click Reboot Compute resource.

    Warning

    Rebooting a compute resource assigned to a data store with a single replica (single-replica compute resource) or degraded virtual disks may result in data loss.


  7. A new screen will open asking for confirmation (via two check boxes) before reboot:

    • Stop all virtual servers that cannot be migrated to another compute resource? Check this box if you want VSs that cannot be migrated to be powered off. When a compute resource is scheduled for a reboot, OnApp will first attempt to hot migrate all VSs it hosts. If hot migration is not possible for a VS, OnApp will attempt to cold migrate that VS. With this box checked, if cold migration fails, the VS will be stopped so the reboot may proceed. If you don't check this box, OnApp will attempt to hot and then cold migrate all VSs hosted by the compute resource being rebooted – but will stop the migration process if any VS cannot be migrated.
    • Are you sure you want to reboot this compute resource? A simple confirmation to confirm that you want the compute resource to reboot.

      Warning

      Before the reboot, please ensure that all vdisks are fully synced and redundant. If some of them are not fully synced, the virtual server, that is owner of a degraded (or non-redundant) vdisk, can loose access to the vdisk. It can be manifested as IO errors during writes or reads to/from the vdisk inside the virtual server.

  8. When you're certain you want to proceed with the reboot, click the Reboot button.

  9. On the compute resource is booted, repair the disk that were degraded during the reboot.

  10. Repeat these steps for all CloudBoot compute resources in your cloud.

  11. Once all CloudBoot compute resources are rebooted, proceed to the Control Panel server upgrade.

 

Anchor
live
live

Live Upgrade

Info
iconfalse
  • Live Upgrade with passthrough is currently unsupported. Passthrough to storage means that network interface will be added to the Storage Controller Server without the bond and the Storage Controller Server will have the complete control over this interface.
  • Power off all Windows virtual machines and virtual backup servers  before starting the live upgrade.

  • If your current Storage package is 4.0, Windows virtual servers can remain running.

  • During the CloudBoot compute resource live upgrade, only the control stack for managing integrated storage is upgraded.  Other changes come into effect after the compute resource is next rebooted. Due to this, hot migration may fail between compute resource which is already rebooted and the one that hasn't.
  • Do not make any changes to the cloud during the upgrade process!
  • Any offline Cloudboot compute resources should be removed from the CP server before running live upgrade as the scripts expect to be able to speak to all compute resources during these steps. 
  • Please, consult OnApp IS Upgrade Paths to learn the minimum Integrated Storage version required for the current update to be performed in LiveUpgrade mode.

 

Use this procedure to upgrade without rebooting your servers:

  1. Make sure no disks are out of sync. To do so, check the Diagnostics page in CP at Dashbord > Integrated Storage > Compute zone label > Diagnostics. Alternatively, log into a compute resource and run the command below: 

    Code Block
    getdegradedvdisks
  2. Repair all the degraded disks before proceeding to the upgrade process. To do so, log in to your CP and go to Integrated Storage > Compute zone label > Diagnostics page. Alternatively, run one of the following commands:

    Code Block
    onappstore repair uuid=
    Code Block
    parallelrepairvdisks
  3. Run the following command from the CP server to stop the OnApp service:

    Code Block
    service onapp stop
  4. Stop the Apache server:

    Code Block
    service httpd stop
  5. Make sure to update CloudBoot packages before proceeding to the following steps.

  6. Run the following command from the Control Panel server terminal to display the list of compute resources with their IP addresses. Make a note of the list of IPs:

    Code Block
    bash#> liveUpdate listHVs 

    This command will also show whether compute resources are eligible for live upgrade.

    Info

    If the command liveUpdate is not available then it may be located in the sbin directory instead (cd /usr/local/sbin).

  7.  Run the following commands from the Control Panel server terminal for each compute resource:

    Code Block
    bash#> liveUpdate updateToolstack <HV IP Addr> 
    Code Block
    liveUpdate refreshControllers <HV IP Addr>
    Info

    Wait several minutes for all degraded disks to come to synchronized state. The synchronization will take approximately three minutes for each compute resource.

    After each controller restart, check for any issues on the backup server (or on one Compute resource from each zone):

    1. Log on via SSH to the backup server (or Compute resource).
    2. Run getdegradednodes from the SSH console.
    3. Run getdegradedvdisks from the SSH console.

  8. Restarts the storage controllers. This command can be performed later at a more suitable time
    Run the following command for each compute resource in turn:

    Code Block
       bash#> liveUpdate restartControllers <HV IP Addr> 
    Note

    Please make sure you restart all controllers and don’t leave your cloud in a partially updated state for too long. Note that when operating in LiveUpdated mode (e.g. with the tool stacks updated but before you have performed the controller restart) you cannot use disk hot plug. 

    Info

    After each controller restart check for any issues on the backup server or one Hypervisor from each zone:
    1. Log on via SSH to the backup server (or Hypervisor).
    2. Run getdegradednodes from the SSH console.
    3. Run getdegradedvdisks from the SSH console.
    If there are any issues seen please rectify them before continuing with the next controller restart.

  9. Make sure that the package versions are upgraded by running the following command on each compute resource:

    Code Block
    bash#> cat /onappstore/package-version.txt | grep Source
  10. Start the Apache server:

    Code Block
    bash#> service httpd start
  11. Start the OnApp service:

    Code Block
    bash#> service onapp start

Anchor
cp
cp

Upgrade Control Panel Server


 

Info
iconfalse
  • CP installer for Installation and Upgrade contains a new -D option enabling to avoid OnApp database dumping during the install/upgrade.
  • To increase the cloud performance we recommend setting RUBY_GC_MALLOC_LIMIT parameter in custom configurations to 16 millions. For more information on RUBY_GC_MALLOC_LIMIT parameter, refer to Ruby’s GC Configuration and Garbage Collection articles.

  • Installer output is redirected to ./onapp-cp-install.log

  • All installer critical errors are in /var/log/messages

  • Custom values must be set before the installer script runs.
  • You may wish to reboot your Control Panel server to take advantage of a new kernel if it is installed. It is not required immediately as a part of the upgrade process though.

 

Excerpt Include
Upgrade Control Panel Server
Upgrade Control Panel Server
nopaneltrue

 

Local Read Policy



Enabling Local Read on a compute zone ensures that the locally stored copy of the data will always be used for reads. This significantly reduces read latency and improves overall storage performance by reducing load on the SAN network. However, in order to use this policy every compute resource must have sufficient physical drives to be able to store the number of stripes specified in the data store. E.g. in a 2R4S data store there must be at least 4 physical disks on the compute resource to use local read.

Changes to Local Read Policy Enforcement


 

Originally, when this policy was introduced OnApp did not enforce the requirement for the minimum number of drives. Consequently, some users who set the policy having insufficient drives may see the following error message:

 

Code Block
Fatal: OnApp::Actions::Fatal Storage API Call failed: {"result"=>"FAILURE", "error"=>"Local reads have been enabled on the zone - members required per host: 4, required hosts: 2, available hosts: 0"}

The solution is to either add additional drives to that compute resource and then add them to the data store or to disable read local.