Blog: The Scale-Out Virtues of Object Storage

File- and block-based storage are well established, but a third alternative, object storage, is hardly new technology. Today, there are countless petabytes of object storage in public clouds, and hundreds of millions of Facebook, Google, and Twitter users routinely rely on the technology. So what is object storage? It’s a method for saving unstructured data in discrete units called objects. Objects are stored in containers in a flat structure rather than the more familiar nested tree structures of file and block storage. Each object is identified by a unique ID, which enables the data to be found without knowing its physical location.

cloud-storage-icon

Whereas most file systems are limited by the number of files, directories, and hierarchical levels they support, object storage can scale almost infinitely. Metadata is kept directly with the object, eliminating the need for lookups in relational databases and further enabling immense scalability. An added benefit is you can easily apply security or retention policies to the metadata.

When should object storage be deployed? The technology is not ideal for transactional or frequently changing data because of performance issues. Block-based solutions provide greater read/write speeds for disk-intensive transactional data. Object storage shines, however, for relatively static data, backups, and cold archival data. Object storage is cost-effective for these applications because it works on commodity hardware.

In a world generating torrents of data every day, the scalability and economy of object storage are attractive, particularly in public and private clouds. What makes this fit even better is object storage pools are accessed via the HTTP protocol or an API, rather than Fibre Channel for block-based storage, for example, or NFS for file-based storage. Sites like Google and Facebook store vast troves of images and videos in object storage.

A key advantage of object storage is how easily it scales. Object storage systems scale out horizontally simply by adding additional nodes. Moreover, because object storage is location agnostic, nodes within a storage pool can be geographically distributed. You can access objects that physically reside on a node anywhere without going through a central controller, delivering a truly global infrastructure in which data can be accessed via the WAN or the Internet. NAS systems also scale out horizontally, but their expansion is restricted by their hierarchical file structures which grow in complexity as they increase in size. Also, NAS systems work best on local networks in the data center. Again, the flat structure of object storage offers almost astronomical scalability for storing unstructured data.

If you need to store petabytes of files that are accessed only occasionally, object-based storage might be your best bet. For large, expanding data troves, such as those found in healthcare, media/entertainment, and other industries, the technology can provide the economical advantages of tape but with less effort and superior retrieval speeds.