Object Storage
Object storage is a computer data storage architecture that manages data as objects, as opposed to other storage architectures like file storage which manages data as a file hierarchy, and block storage which manages data as blocks within sectors and tracks.
Note: "Object storage is an alternative to NAS for handling unstructured data. There is speculation that object storage gradually will overtake scale-out NAS, but it's also possible the two technologies will continue to survive side by side. Both storage methodologies deal with scale, only in different ways. Object storage surfaced as a new method for easily scalable storage in web-scale environments. It often encompasses unstructured data that is not easily compressible, particularly large video files."
Examples...
Block: NVMe, NVMe-oF, iSCSI, Fibre Channel
Parallel file: Lustre, Spectrum Scale, Panasas, WekaIO, BeeGFS
Object: S3 (RESTful API)
Block Storage (SAN / "Structured")
Virtual servers, databases, etc...
Best suited for transactional data and frequently changing data.
Protocols: SCSI, Fibre Channel, SATA
Low Latency / High Performance.
Connected via FC and/or iSCSI.
Can be very expensive.
Limited Scalability.
Difficult to extend past the data center.
File Storage (NAS / "Unstructured")
Higher Latency / Higher Throughput.
Best suited for shared file data.
Protocols: CIFS and NFS
Connected via SMB, NFS, etc... and accessed via LAN.
Not as expensive as block.
Limited Scale Out.
Complex to manage at scale.
Difficult to extend past the data center.
Object Storage ("Unstructured")
Best for accessibility and reliability at scale. That scalability is the direct result of object storage's shared-nothing architecture and flat structure.
Storage backbone of the cloud.
Great for large sets of unstructured data.
Best suited for relatively static file data and as cloud storage.
Protocols: REST and SOAP over HTTP. The defacto standard RESTful API is currently S3
Ability to connect any type of device from anywhere.
Supports protocols like HTTP.
Parallelization becomes a key characteristic.
Object storage leverages metadata. There is no hierarchy to scan or crawl.
Millions of devices can access the same information simultaneously.
NOTE: Erasure coding is great for data resilience, but it adds significant latency for reads and writes. Erasure coding also doesn't protect against data corruption, human errors or malware/ransomware. Object storage, in general, doesn't provide snapshots, a staple for block and file storage that does provide that level of protection.
Object Storage examples: AWS S3, Azure Blob, Google Storage, Wasabi, MinIO
AWS Object Storage – Multiple performance and price points, including standard, auto-tiering, infrequent access, and archive.
Azure Object Storage – Premium, hot, cool, and archive performance tiers, each with pay-as-you-go or reserved capacity purchase options. Volume discounts apply across all objects.
Google Object Storage – Four performance tiers – Standard, Nearline, Coldline, and Archive
Object storage systems allow retention of massive amounts of unstructured data. Object storage is used for purposes such as storing photos on Facebook, songs on Spotify, or files in online collaboration services, such as Dropbox. Object storage maintains availability by storing multiple copies of data over multiple nodes through a replication process. If one node fails, the data is still available, and no downtime is experienced.
File storage organizes and represents data as a hierarchy of files in folders
Block storage chunks data into arbitrarily organized, evenly sized volumes;
Object storage manages data and links it to associated metadata.
Advantages of Object Storage...
Data Protection.
Searching, Indexing and Metadata.
Performance.
Security.
Compliance & Auditing.
Deployment Models.
Protocol Support and Standards.
You can't use object storage services to back a traditional database, due to the high latency of such services. Object storage doesn't allow you to alter just a piece of a data blob, you must read and write an entire object at once.
The storage can be addressed with a few commands:
PUT: creates an object
GET: reads an object
DELETE: deletes an object
LIST: lists all objects
NEW! Veeam: How to deploy QNAP QuObjects to have an Object Storage on-prem and use it as Capacity Tier
Why is SSD not a good option for long-term storage/archives?
"Long term? No. SSDs store data in “floating gate transistors.” These transistors have a gate surrounded by insulator. A weird quirk of quantum physics means you can get electrons to “tunnel” onto the gate, passing onto the gate without crossing the space between, and then, since the gate is surrounded by insulator, the electrons remain trapped there.
However, the same weird quirk of quantum physics means that every now and then, an electron will tunnel right back out.
If you take an SSD, record files on it, and sit it on a shelf, over time the data on the disk will slowly evaporate. Come back in five years and the disk may be unreadable and all the files corrupt.
Summary: Don't do this"