swift object storage

Object storage features

Software to reliably store billions of objects distributed across standard hardware.

  • It’s not a file system
  • Fully distributed evenly throughout the system
  • Account/Container/Object structure
  • No Central Database
  • Access with RESTFUL API
  • Auditors to check staleness of data
  • Provides data redundancy (at least 3 replicas)
  • Runs on commodity hardware

User view: Accounts, Container, Objects

User access an account account owns containers, containers contain objects

Containers and Objects

Container  

  • A Storage compartment for your data
  • Data must be stored in container
  • Like a folder in Windows or directory    in Linux.
  • Can create an unlimited number of  containers within your account
 

Object:

  • Basic storage entity + Optical meta data
  • When you upload data to openstack object storage, the data stored as-is
  • Uploaded data Consists of A location, Object name, The data, Optional user specified meta data consisting of key/value pairs.

The swift Architecture is very distributed to prevent any single point of failure as well as to scale horizontally. It includes the following components.

  • Proxy server accepts incoming requests via the openstack API. It accepts files to upload, modifications to metadata or container creation. The proxy server may utilize an optional cache (memcache) to improve performance.
  • Account servers manage accounts defined with the object storage service.
  • Container servers manage mapping of containers (i.e folders) with in the object store service
  • Object servers manage actual objects (i.e files) on the storage nodes.
  • There are also number of periodic process which run to perform housekeeping tasks. The most important of these is the replication services.
 

 

Ring Arch 

  • Maps account/container/object to partition
  • Divided into partitions, each  addressable by an index value.
  • 3 Rings: Account, Container, Object 
  • Data replicated across nodes-to avoids SPOF

 Ring properties 

Determines where the data should reside in the cluster.

  • Maintains this mapping using zones, devices, partitions, and replicas.
  • A copy of the Ring should reside on every node of the cluster

Rings are statically built and assigned as part of post installation tasks

  • Rings uses a gzipped data structure

Separate Ring for Account database, Container database, objects

  • The servers do not modify the rings – they are externally managed.
  • Built and Managed by ring builder utility.

Helpful CLI

Testing using CLITesting using CLI

Swift stat

Finding an account, container, or object

Swift-get-nodes

Display object information

Swift-object-info

Crawl the account, checking that all containers and objects can be found.

Swift-account-audit