The Forgotten Factor of Cloud Memory Organization
With practically unlimited data storage possibilities and access from anywhere you have an Internet connection, does it really matter how cloud memory works? Actually, yes, it does matter. While nobody wants to be forced to dive into the gory detail of it all, a few basic principles help to understand when to choose cloud storage and when it can be a challenge.
That Obscure Object of Desire
Cloud memory is typically organized to store and manage ‘objects’. This is more like how people remember things and less how older computers handled data. Here’s the difference. Traditional Desktop PCs and servers stash data away in blocks, which naturally enough is called Block Storage. These blocks are all the same size and hold the same amount of data. Let’s say you want to store a photo. Depending on the size of the photo, you’ll need a certain number of blocks, each holding a bit of your photo. You’ll also need to track where all those blocks are on your hard disk to make sure you get all of your photo back when you want it. By comparison, object storage doesn’t bother about chopping your photo up into same-sized chunks. It just stores your photo as one big lump – done!
Good Things about Object Storage
Using object storage for cloud data repositories wasn’t an accident. It was a deliberate choice, because it fitted better with the new ways people wanted to handle all the extra information that was being generated – images, videos, unstructured text, and big data in general. In particular, users needed:
- Scalability. Conventional block-oriented systems hit limits much earlier than object storage systems that are essentially simpler and easier to manage.
- Flexibility. Grab, store and recover a big ‘lump’ of data directly as it is, together with any extra information on the data (‘metadata’ such as the age, type, quantity, and owner of the data.)
- Reliability. Object storage lends itself well to data replication to guard against data loss or corruption through data ‘rot’ and system failures.
- Affordability. Object storage can be done well on commodity (OK, cheap) IT systems.
If these four characteristics together say ‘cloud’ to you, then you’re bang on track. And that’s why object storage accounts for so much of all cloud service and system activity.
Your Honor, I Object!
Up in front of the Internet judge and jury, does object-oriented storage always win against block-oriented storage? Not so fast – there are also objections against the use of object-oriented storage. For example, even if you want just a bit of your object back, you still have to have it all back at once. By comparison, block-oriented storage lets you pick out the precise parts that interest you. This is a reason why in absolute performance terms block storage can outdo object storage. As a general guide, object storage suits web content, media content, backups, archives and similar ‘blobs’. Block storage is a better choice for structured databases and other data that requires random access.
Object and Block Storage Coexistence
Can we work something out and have the best of both data storage systems? The OpenStack suite of software for building standard cloud services includes both models. Cloud memory systems themselves convert block-oriented data sent from devices ‘down below’ into the objects they use and understand. This conversion takes extra time and effort. However new file transfer protocols are building in this conversion to be able to efficiently map data directly onto the cloud object format as it arrives, with no further conversion required.
In short, remember the basics about these two major ways of storing data, and you’ll be able to get that much more out of your cloud service.
