Technology & Architecture

Technology

Scality’s technology is in part an evolution of some revolutionary ideas described in a few key academic texts and research papers that address the issues of massive scalability in storage environments. From the earliest conceptual design phase of RING, there has been a determination that certain technological characteristics were essential – such as P2P topology, Distributed Hash Tables (DHTs), efficient data placement, fast inter-node routing protocols and specific key format and mapping techniques.

Two patents that contribute to Scality’s uniqueness:

1) Multipurpose storage system based upon a distributed hashing mechanism with transactional support and failover capability (US and WIPO patent WO/2010/080533)
2) Probabilistic offload engine for distributed hierarchical object storage (US and WIPO patent WO/2011/072178)

Two key ingredients in the technology are:

1) The Chord protocol invented at MIT
2) The key format invented by Scality

The Chord protocol invented at MIT:

This is a true second generation P2P mechanism. Scality RING applies this protocol by organizing servers in a ring topology that defines the address space. The routing protocol avoids broadcasting messages to all nodes in a cluster and establishes a structured layout. With Chord, each node just needs the knowledge of its successor and predecessor nodes. This is especially fundamental, as the cluster could be very large in terms of the number of nodes, and can potentially change with the addition and removal of servers.

The key format invented by Scality:

This key is 160 bits with 128 bits for the object ID, 24 bits for the dispersion and 8 bits for the class of service.

The dispersion field guarantees that there are no collisions between objects and their copies. That is to say, an object will not be stored on the same disk, server or rack as its replica, thereby affording a greater level of durability in the event of a failure.

The class of service defines the protection policy (for example, by replication or erasure code, or by the number of replicas or data and coding parts) to be applied to the specific object.

Generated keys are organized in a ring with a consistent hashing formula that accommodates node insertion (joins) and removal (failures or disengagements/ring "leaves") without the need to broadcast all changes to all nodes. When a node is inserted in ring with n nodes, only a portion of the key space must be remapped.

Central to this architecture is the absence of any central intelligence that might constitute a single point of failure. Everything is distributed. Any node can receive a request from outside the RING, and the path to the right node is described by ½ log2(n), with n being the number of nodes. This lookup is very efficient, and has a linear relationship with the number of nodes during the lifetime of the configuration.

Core RING features such as Replication, ARC, Auto-Tiering and Geographic redundancy  are run from within the cluster by the storage nodes themselves.

Another key differentiator of RING is MESA. RING makes extensive use of this internally developed NewSQL distributed database. Used for the control and management or the RING and to store server metadata, this database is fundamental to RING’s scalability and high performance.

RING configurations are extremely flexible:

RING can be configured to accommodate extremely high availability and partition tolerance (AP in Brewer nomenclature), or to support strict consistency with partition tolerance (CP).

For more detailed information, please see the RING technical white paper.

Architecture

Scality RING architecture starts with traditional servers grouped together in racks to form a cluster. Comprised of three different elements, the architecture includes servers, connectors and a Supervisor.

Servers

A physical server is divided into logical storage nodes connected to IO daemons (iods). These iods run in parallel and are responsible for their own dedicated storage units. Storage IOs are always performed via iods. A maximum of 255 iods can be configured per server.

This aspect of the architecture makes an important contribution to the parallelism of the overall infrastructure. Parallelism is defined by the number of storage nodes running on a server multiplied by the number of iods. Today, with the increasing density of servers, this ability is becoming an essential feature, and is a fundamental differentiator in the market.

By default, RING is configured to run with 6 storage nodes on each server. The number of iods depends on the number of disks available within the server. As a result, given the hardware resources on a server, the more storage nodes and iods you define, the better the request response time and throughput.

The generated keyspace incorporates configuration elements, such as replication schemes and the number of data centers, then maps them to the different storage nodes to form a RING. For more information about the key format, please see the description above.

Connectors:

Connectors act as the middlemen between applications and storage nodes. [1] The connectors play an essential role in data replication.

Scality offers a variety of different connectors that can be deployed on the application servers, on dedicated servers or on the RING nodes themselves.

Scality connectors include:

• REST/HTTP-based connectors, such as SRWS, RS2 Light and the fully S3-API compatible RS2, useful for building a comprehensive STaaS service
• Email connectors with support for multiple email systems, such as VMware, Zimbra, Openwave, Open-Xchange, Critical Path, Cyrus or Dovecot
• Scale Out File System connectors to interact at the file level
• Dedicated Digital Media connectors to support origin servers in a CDN environment

Supervisor:

The Supervisor is installed on a dedicated machine. Through its Web GUI, it provides the full control and management of the deployed RING infrastructure. An alternative method is the use of the RingSH command line interface (CLI).