Storage

Any SAN that can present a block device: iSCSI, ATAoE, Fiber.

Environment

OnApp with SAN Storage

Answer

It depends on what the virtual servers are going to be used for and target performance. The more IOPS the better as far as your budget allows.

The root cause here is that you archive the statistics on your cloud. Therefore, archive statistics remove your old statistics to clean up space on the database.

Yes, but this isn't how a cloud should really be designed. We can make local storage work for you, but bear in mind that there won't be any failover, because storage would be tied to just one compute resource.

Fibre SAN is supported as long as the device itself is supported by RHEL5. Compute resources can create virtual servers on a shared LUN.

OnApp Storage is a distributed SAN for cloud providers. It turns your existing cloud infrastructure into an extremely scalable, resilient and efficient storage system that is optimized for cloud workloads.

OnApp Storage is an extremely cost-effective cloud storage platform that uses commodity hardware. There is no vendor lock-in with supported disk types or custom network backplanes: you can use off-the-shelf compute resource servers, disks and Ethernet components. You can build your distributed SAN with SATA, SAS or SSD devices, and get exceptional performance even in a 1Gbit network environment. You deploy additional disk drives in the compute resource servers used by OnApp Cloud, and assign them to virtual data stores. Data stores are categorized by performance, and have their own policies for striping, redundancy and over-commit. OnApp Storage has a modular design - you can add additional storage capacity when you need it, without having to rebuild the whole SAN.

OnApp Storage has a number of technical and commercial requirements that must be met to ensure correct performance.

Use commodity and pre-existing hardware
Use existing storage (HDD/SSD/SCSI) on existing compute resources.

Performance to price ratio in comparison to dedicated SAN
Performance must be comparable to that of dedicated SAN systems, using similar hardware.

Scalability
Use distributed techniques where possible and a system that avoids single points of failures.

Resilience
Allow for multiple replicas of data and also multiple points of access to the system.

Efficiency
System overheads should be within an acceptable limit in terms of additional resources.

More on OnApp Storage Architecture.

Backend nodes can only host up to a max of 128 vDisks.

The number of NBD paths available for your virtual disks depends on the amount of RAM available to the storage node. You can use the following formula to calculate the maximum number of NBD paths:

N = (Controller memory size - 128) ÷ 4, where:
Controller memory size = the memory assigned to the storage controller (the default is 1024MB).
128 = amount of system memory reserved for the storage controller
4 = the amount of memory needed per NBD server

So by default:
(1024 -128) = 896 MB available for NBD servers
N = 896 ÷ 4 = 224 NBD paths

Each stripe in a datastore needs 1 NBD path, so the total number of vDisks, D, you can have is given by:
D = N ÷ (SxR),
where:
S = the number of stripes in the datastore
R = the number of replicas

For example:
If the data store has 2 replicas and 2 stripes, it requires 4 paths (2 strips x 2 replicas) per disk. Using the defaults for the storage controller each controller can host:
224 ÷ 8 = 56 vDisks

Linux virtual servers have at least 2 disks (main disk and swap disk) so 8 paths would be required (if you are using the same data store configuration for main disk and swap). So this configuration supports 28 Linux virtual servers or 56 Windows virtual servers. To fit more virtual servers in the cloud, we recommend using unreplicated storage for swap drives.

If you are experiencing MAC address flapping across ports because the switch does not support the balance-rr mode, set up separated VLANS per each bond pair for that switch.

There is no technical issue with mixing different size disks in the same data store. However we would suggest that you try to keep drives balanced between compute resources if redundancy is required. VDisks (Virtual Disks) will be created according to the data store configuration and look to spread stripe sets (replicas) over different compute resources so that if a compute resource goes down another should be able to run the content after the VS is migrated (if it was hosted on the compute resource).

It is common practice to create several data stores and normally group the disk drives based on performance levels (e.g. keep SSD separate from SATA to avoid wasting IO throughput for synchronous writes).

Administration is made slightly more complex with different size disk drives: some operations, such as backup and resize all of the disk members for a VDisk, will need to be able to support the operation. Otherwise the transaction will roll back.

Normally, compute resources are separated into groups of 10. Each compute resource is limited to a number of disk drives by the following calculation:
4 default disks per controller VS
640MB default RAM. For performance of the network block devices we suggest increasing the default RAM to 2GB. This means that 2GB of RAM and one vCPU will be assigned per storage controller per every 4 disk drives. We have seen clients with 20+ disk drives in a single compute resource with no reported issues.

We are currently working on improving this part of the product based on customer feedback. We are implementing a customizable alerting and monitoring system that will warn Cloud owners when the disk drives are approaching set limits. We are also working on implementing a semi-automated system that will rebalance content to disk drives with more free space.

For example, you have two virtual servers, their disks have the same size, but backups take much longer on one of them than on the other. As OnApp backs up the used space on the disks for virtual servers, if one virtual server has 10 GB used and the other 40 GB used, the latter is likely to take much longer for the backup to complete.

40 GB is reserved to ensure your data store does not get completely used. If the data store is filled up, there is no required storage for snapshot backups to complete successfully.