Here Today, Hybrid Cloud Tomorrow? Prepare for Data Storage and Transfer Challenges
Resilient, flexible, scalable and pay as you go are the attributes of the public cloud. That makes it an immediate candidate for offloading data from conventional IT or private cloud installations, and for storing backup data. But does that also make it a good candidate? Blindly transferring data without paying attention to data criticality and link speeds could cause problems down the line, especially if you need to bring the data back to your premises in a hurry.
What Should Go and What Should Stay?
Tiered data storage locally has been in existence for some time. The often-used, mission-critical files stay on disk, while the ones destined for archiving or for only infrequent consultation get sent off to tape (for instance). Adding public cloud into the mix gives you another tier to deal with, or rather, for backup appliances to deal with. Smart solutions will facilitate sorting data into these different categories or tiers; they will recognize that you’ll probably want the essential data to be immediately available on site with the guaranteed high speed of a LAN connection (as opposed to the more dubious performance of some Internet connections).
The Spillover Conundrum (Apps in New York, Data in Miami)
Some operational models are set up to move application data out to the cloud when predefined limits are exceeded. This spillover or overflow model is attractive because of the virtually limitless capacity of the cloud for any one customer. An enterprise doesn’t have to worry about provisioning or running out of storage space, or of losing application data. However, transfer speed needs to be taken into account. The problem is not so much in the storage of excess data, but in use of the data (now in the cloud) by the originating application (still running locally). Speed, latency and jitter of the reverse data transfer can degrade local application performance. In some instances where microseconds count, financial trading for example, a hybrid model may simply not yet be good enough.
The Memory Map and the Cloud Territory
If the points above weren’t enough to deal with, there’s more. Cloud storage is organized by object, but still uses FTP or HTTP protocols, both of which are based on Internet’s good old TCP. Files from local servers have to be decomposed into many smaller parts in order to be stored as objects in the cloud. When TCP flow control detects delays because of the time needed for this processing, it can slow file transfer from a flood to a trickle. Under the best circumstances, the maximum transfer rate of a large file from local server to cloud is around 100 megabytes per second. But that’s not enough with the terabyte-sized files some organizations want to move backwards and forwards.
Accelerate or Avoid? Two Different Solutions for Hybrid Cloud File Transfer and Storage
One solution is available in the form of a file transfer protocol called FASP (Fast and Secure Protocol). This avoids the TCP flow control problem and handles files in a way that makes them apt for storage in the cloud immediately with no breaking down or conversion required. Estimates of transfer speed improvements vary between a factor of 5 or of 100 or more. Another solution is to get your back-up files into the cloud (FASP or no FASP) – and leave them there. Now that Disaster Recovery as a Service (DRaaS) is a reality, organizations can switch to cloud mode for their operations until the local disaster has been resolved. Then comes the interesting problem of switching from the cloud DRaaS mode back to local server mode, but we’re saving discussions on that one for a future blog post.
