Important Notes
- You must be running the latest patch of OnApp 4.1 version to upgrade to 4.2 version. If you are using an earlier version, please upgrade to 4.1. first.
- Check the Activity Log in your OnApp CP dashboard if there are no transactions running in your cloud. If so, wait until all transactions are complete.
Make sure no Control Panel files are open for editing under the root user account.
If you are using a third-party billing platform, please ensure that this is compatible with OnApp 4.2 before proceeding with the upgrade! The latest WHMCS modules can be found here.
If you are using WHMCS modules, make sure to update the PHP Wrapper after you update OnApp Cloud. Download the latest wrapper.
We strongly recommend that you test all your custom scripts before upgrading your production environment.
- To avoid VNC console issues, make sure that ServerName Apache settings match the SSL certificate.
- Note that if you are using Floating IPs in your environment or if you have VS with primary IPs which could respond to your Control Panel server from elsewhere on your network we would recommend to disable the new 'Ping hosted virtual servers before initiating failover' setting to avoid the possibility of a false-positive ICMP result.
- Be aware that OnApp does not support UEFI on static compute resources. You should disable UEFI on your compute resources before installing OnApp.
- Drives assigned for use by Integrated Storage are identified using a disk signature that is generated using SCSI page query mechanism to the device. Please note that disk signatures may change across different kernel versions following an upgrade and reboot. If this occurs, go to the compute resource edit page to re-identify and select the correct drives. Please contact support if you have any concerns regarding this operation.
- If you are an Accelerator Beta customer, please contact OnApp support to discuss upgrading.
Upgrade Control Panel Server
To upgrade your Control Panel server: Stop the Monit service: Run the following command from the CP server to stop the OnApp service: Download and install the latest OnApp YUM repository file: Upgrade OnApp Control Panel installer package: Update your server OS components (if required): (Optional) If you need some custom Control Panel configuration, set the values before the installer script runs. If the Run Control Panel installer: Customers running vCloud integration, run Control Panel installer with specified rake task: You may wish to reboot your Control Panel server to take advantage of a new kernel if it is installed. It is not required immediately as a part of the upgrade process though. Customers running vCloud integration pay attention that upgrade process can take some time, as vCloud and OnApp synchronization is running along with OnApp upgrade. Perform the following steps if you plan to deploy Accelerator. Otherwise skip. See the installer options below for details. Specify user name and password for rabbitmq-server: Compute Resources and Control Panel must use the same rabbitmq-server. # service monit stop
service onapp stop
bash#> rpm -Uvh http://rpm.repo.onapp.com/repo/onapp-repo-4.2.noarch.rpm
bash#> yum update onapp-cp-install
bash# /onapp/onapp-cp-install/onapp-cp-install.sh -y
bash# vi /onapp/onapp-cp.conf
onapp-cp.conf
file is not configured correctly, it will replace the SSL files with a self-signed even if a legitimate certificate is already installed.bash#> /onapp/onapp-cp-install/onapp-cp-install.sh
# /onapp/onapp-cp-install/onapp-cp-install.sh --rake='vcloud:resync'
rabbitmqctl add_user username 'userpass'
rabbitmqctl set_permissions -p '/' username ".*" ".*" ".*"
service onapp restart
To upgrade the OnApp Storage packages: Upgrade the repo: Upgrade the packages: Run the script: Be aware that the disk-less nodes password is the root password for the CloudBoot compute resources. By default it is blank. When run in the interactive mode, enter the required information. Make sure to update CloudBoot packages before proceeding to the upgrade of CloudBoot backup servers. CloudBoot backup servers are CloudBooted KVM compute resources that can be be used as backup servers. The CloudBoot backup server upgrade procedure is almost the same as the CloudBoot compute resource upgrade. Follow the instructions provided in this section to upgrade CloudBoot backup servers in your cloud. Once you have upgraded the CloudBoot dependencies, you have to reboot your Cloud Boot compute resource to update the Cloud Boot RPM. You do not need to perform any backup server upgrade operations using console. To do so: Go to your Control Panel Settings menu. Click the Compute resources icon. Click the label of the CloudBoot compute resource the backup server is based on. On the compute resource details screen, click the Actions button, then click Reboot Compute resource. When you're certain you want to proceed with the reboot, click the Reboot button. Repeat these steps for all CloudBoot backup servers in your cloud. Once all are rebooted, proceed to CloudBoot compute resources upgrade. Depending on the infrastructure, scale and needs of your cloud we suggest the following methods of upgrading CloudBoot Compute resources: In case you have applied any custom configuration to your CloudBoot servers, it is recommended to recheck that this customization does not break new cloud boot image version. For this, reboot a compute resource and run Storage Health Check and Network Health Check. Make sure that Vdisks hosted on a compute resource are redundant and healthy before rebooting a CloudBoot compute resource. Follow the below procedure to upgrade the CloudBoot compute resources with reboot: 3. Reboot all CloudBoot compute resources. Note that virtual servers cannot be stopped simultaneously, but must be stopped in sequence. This can result in considerable downtime if there are a large number of virtual servers. Use this procedure if you prefer migrating all virtual servers to another compute resource and conducting overall upgrade of your CloudBoot and Integrated Storage. Virtual servers that do not support hot migrate will have to be stopped. Once you have upgraded the CloudBoot packages, you have to reboot your CloudBoot compute resources to update them. To do so: Run the following command from the Control Panel server terminal to display the list of compute resources with their IP addresses. Make a note of the list of IPs: This command will also show whether compute resources are eligible for live upgrade. If the command liveUpdate is not available then it may be located in the sbin directory instead (cd /usr/local/sbin). Run the following command for every compute resource: Once all the toolstacks are updated run the following command for every compute resource: Wait several minutes for all degraded disks to come to synchronized state. The synchronization will take approximately three minutes for each compute resource. After each controller restart, check for any issues on the backup server (or on one Compute resource from each zone): 1. Log on via SSH to the backup server (or Compute resource). Migrate all the virtual servers from the CloudBoot compute resource to another compute resource. Follow the instructions described in the Migrate Virtual Server section of the Admin guide to migrate virtual servers. After that, go to your Control Panel Settings menu. Click the Compute Resources icon. Click the label of the CloudBoot compute resource you have migrated all VSs from. On the compute resource details screen, click the Actions button, then click Reboot Compute resource. Rebooting a compute resource assigned to a data store with a single replica (single-replica compute resource) or degraded virtual disks may result in data loss. Are you sure you want to reboot this compute resource? A simple confirmation to confirm that you want the compute resource to reboot. Before the reboot, please ensure that all vdisks are fully synced and redundant. If some of them are not fully synced, the virtual server, that is owner of a degraded (or non-redundant) vdisk, can loose access to the vdisk. It can be manifested as IO errors during writes or reads to/from the vdisk inside the virtual server. When you're certain you want to proceed with the reboot, click the Reboot button. Once the compute resource is booted, repair the disk that were degraded during the reboot. Make sure no disks are out of sync. To do so, check the Diagnostics page in CP at Dashbord > Integrated Storage > Compute zonelabel > Diagnostics. Alternatively, log into a compute resource and run the command below: Repair all the degraded disks before proceeding to the upgrade process. To do so, log in to your CP and go to Integrated Storage > Compute zonelabel > Diagnostics page. Alternatively, run one of the following commands: Repeat these steps for all CloudBoot compute resources in your cloud. Live Upgrade is only applicable if your cloud is running latest 4.2 CloudBoot RPM. Use this procedure to upgrade without rebooting your servers: Make sure no disks are out of sync. To do so, check the Diagnostics page in CP at Dashbord > Integrated Storage > Compute zonelabel > Diagnostics. Alternatively, log into a compute resource and run the command below: Repair all the degraded disks before proceeding to the upgrade process. To do so, log in to your CP and go to Integrated Storage > Compute zonelabel > Diagnostics page. Alternatively, run one of the following commands: Run the following command from the CP server to stop the OnApp service: Stop the Apache server: Make sure to update CloudBoot packages before proceeding to the following steps. Run the following command from the Control Panel server terminal to display the list of compute resources with their IP addresses. Make a note of the list of IPs: This command will also show whether compute resources are eligible for live upgrade. If the command liveUpdate is not available then it may be located in the sbin directory instead (cd /usr/local/sbin). Run the following command for every compute resource: Once all the toolstacks are updated run the following command for every compute resource: Wait several minutes for all degraded disks to come to synchronized state. The synchronization will take approximately three minutes for each compute resource. After each controller restart, check for any issues on the backup server (or on one Compute resource from each zone): 1. Log on via SSH to the backup server (or Compute resource). Restarts the storage controllers. This command can be performed later at a more suitable time. Please make sure you restart all controllers and don’t leave your cloud in a partially updated state for too long. Note that when operating in LiveUpdated mode (e.g. with the tool stacks updated but before you have performed the controller restart) you cannot use disk hot plug. After each controller restart check for any issues on the backup server or one Hypervisor from each zone: Make sure that the package versions are upgraded by running the following command on each compute resource: Start the Apache server: Start the OnApp service: Perform the following steps for your Cloudboot compute resources if you plan to deploy Accelerator. These steps are to be performed on each of the compute resources. Copy file: Open Change owner: Run the following: Note that steps 1-4 of the above instruction should be done after every reboot of CloudBoot compute resource. You can run the following commands (using your parameters) to the custom config instead: Also you can monitor RabbitMQ server using a command-line tool for it. Upgrade CloudBoot Packages
CP_host#> rpm -Uvh http://rpm.repo.onapp.com/repo/onapp-repo-4.2.noarch.rpm
CP_host#> yum update onapp-store-install
CP_host#> /onapp/onapp-store-install/onapp-store-install.sh
Upgrade CloudBoot Backup Servers
Upgrade CloudBoot Compute Resources
Simple Reboot This method is the simplest method technically. It also ensures all tools are updated. However, it will result in some limited downtime (its duration depends on how many virtual servers are running on each compute resource). Migrate and reboot This method involves migrating all virtual servers off each CloudBoot compute resource in turn. The compute resource can then be safely rebooted, picking up the upgraded Integrated Storage and CloudBoot packages. Virtual servers that do not support hot migrate will have to be stopped. Live Upgrade This method will upgrade Integrated Storage components but will not upgrade CloudBoot image. Simple Reboot
Migrate and reboot
CP_host#> liveUpdate listHVs
CP_host#> liveUpdate updateToolstack <HV IP Addr>
CP_host#> liveUpdate refreshControllers <HV IP Addr>
2. Run getdegradednodes from the SSH console.
3. Run getdegradedvdisks from the SSH console.HV_host#> getdegradedvdisks
HV_host#> onappstore repair uuid=
HV_host#> parallelrepairvdisks
Live Upgrade
HV_host#> getdegradedvdisks
HV_host#> onappstore repair uuid=
HV_host#> parallelrepairvdisks
CP_host#> service onapp stop
CP_host#> service httpd stop
CP_host#> liveUpdate listHVs
CP_host#> liveUpdate updateToolstack <HV IP Addr>
CP_host#> liveUpdate refreshControllers <HV IP Addr>
2. Run getdegradednodes from the SSH console.
3. Run getdegradedvdisks from the SSH console.
Run the following command for each compute resource in turn: CP_host#> liveUpdate restartControllers <HV IP Addr>
1. Log on via SSH to the backup server (or Hypervisor).
2. Run getdegradednodes from the SSH console.
3. Run getdegradedvdisks from the SSH console.
If there are any issues seen please rectify them before continuing with the next controller restart.HV_host#> cat /onappstore/package-version.txt | grep Source
CP_host#> service httpd start
CP_host#> service onapp start
Configuration for Accelerator
cp /home/mq/onapp/messaging/credentials{_example,}.yml
vi /home/mq/onapp/messaging/credentials.yml
and check the following details:---
host: 10.0.50.4 # RABBITMQ SERVER IP/FQDN
port: 5672 # RABBITMQ CONNECTION PORT(default: 5672)
vhost: '/'
user: accelerator-example # RABBITMQ USER NAME
password: 'e{y31?s8l' # RABBITMQ ACCESS PASSWORD
queue: 'hv-10.0.50.102' # hv-[IP Address of Compute Resource]
exchange:
name: 'acceleration'
type: 'direct'
durable: True
chown -R mq:mq /home/mq
service onapp-messaging start
cp /home/mq/onapp/messaging/credentials{_example,}.yml
echo "---
host: 10.0.50.4 # RABBITMQ SERVER IP/FQDN
port: 5672 # RABBITMQ CONNECTION PORT(default: 5672)
vhost: '/'
user: accelerator-example # RABBITMQ USER NAME
password: 'e{y31?s8l' #RABBITMQ ACCESS PASSWORD
queue: 'hv-10.0.50.102' # hv-[IP Address of Compute Resource]
exchange:
name: 'acceleration'
type: 'direct'
durable: True" > /home/mq/onapp/messaging/credentials.yml
chown -R mq:mq /home/mq
service onapp-messaging restart
Local Read Policy
Enabling Local Read on a compute zone ensures that the locally stored copy of the data will always be used for reads. This significantly reduces read latency and improves overall storage performance by reducing load on the SAN network. However, in order to use this policy every compute resource must have sufficient physical drives to be able to store the number of stripes specified in the data store. E.g. in a 2R4S data store there must be at least 4 physical disks on the compute resource to use local read.
Changes to Local Read Policy Enforcement
Fatal: OnApp::Actions::Fatal Storage API Call failed: {"result"=>"FAILURE", "error"=>"Local reads have been enabled on the zone - members required per host: 4, required hosts: 2, available hosts: 0"}
The solution is to either add additional drives to that compute resource and then add them to the data store or to disable read local.
Getting support for your upgrade
You can use the information in this document to perform your own upgrade to the 4.2 version of the OnApp Cloud. However, if you have a full OnApp Cloud license, you are entitled to free upgrade support from the OnApp Support team.
If you would prefer to have the Support team perform the upgrade for you, just raise a ticket in the normal way. Please be aware, however, that there may be a queue! For help with your upgrade, visit the OnApp community forum: http://forum.onapp.com.