Object Storage

Object storage is a computer data storage architecture that manages data as objects, as opposed to other storage architectures like file storage which manages data as a file hierarchy, and block storage which manages data as blocks within sectors and tracks.

• New: Blog Series: Cloud Object Storage Deep Dive - @micoolpaul

Note: "Object storage is an alternative to NAS for handling unstructured data. There is speculation that object storage gradually will overtake scale-out NAS, but it's also possible the two technologies will continue to survive side by side. Both storage methodologies deal with scale, only in different ways. Object storage surfaced as a new method for easily scalable storage in web-scale environments. It often encompasses unstructured data that is not easily compressible, particularly large video files."

Examples...

• Block: NVMe, NVMe-oF, iSCSI, Fibre Channel

• File: NFS, SMB

• Parallel file: Lustre, Spectrum Scale, Panasas, WekaIO, BeeGFS

• Object: S3 (RESTful API)

Block Storage (SAN / "Structured")

• Virtual servers, databases, etc...

• Best suited for transactional data and frequently changing data.

• Protocols: SCSI, Fibre Channel, SATA

• Low Latency / High Performance.

• Connected via FC and/or iSCSI.

• Can be very expensive.

• Limited Scalability.

• Difficult to extend past the data center.

File Storage (NAS / "Unstructured")

• Higher Latency / Higher Throughput.

• Best suited for shared file data.

• Protocols: CIFS and NFS

• Connected via SMB, NFS, etc... and accessed via LAN.

• Not as expensive as block.

• Limited Scale Out.

• Complex to manage at scale.

• Difficult to extend past the data center.

Object Storage ("Unstructured")

• Best for accessibility and reliability at scale. That scalability is the direct result of object storage's shared-nothing architecture and flat structure.

• Storage backbone of the cloud.

• Great for large sets of unstructured data.

• Best suited for relatively static file data and as cloud storage.

• Protocols: REST and SOAP over HTTP. The defacto standard RESTful API is currently S3

• Ability to connect any type of device from anywhere.

• Supports protocols like HTTP.

• Parallelization becomes a key characteristic.

• Object storage leverages metadata.  There is no hierarchy to scan or crawl.

• Millions of devices can access the same information simultaneously.

• NOTE: Erasure coding is great for data resilience, but it adds significant latency for reads and writes. Erasure coding also doesn't protect against data corruption, human errors or malware/ransomware. Object storage, in general, doesn't provide snapshots, a staple for block and file storage that does provide that level of protection.

Object Storage examples: AWS S3, Azure Blob, Google Storage, Wasabi, MinIO

• AWS Object Storage – Multiple performance and price points, including standard, auto-tiering, infrequent access, and archive.

• Azure Object Storage – Premium, hot, cool, and archive performance tiers, each with pay-as-you-go or reserved capacity purchase options. Volume discounts apply across all objects.

• Google Object Storage – Four performance tiers – Standard, Nearline, Coldline, and Archive