S3 to a Waiting World

SOLVED MAGAZINE

Explore Scality's new online magazine where you will find all the latest articles and news.

As the clever name suggests: AWS S3 (Simple Storage Service) is simple, and who isn’t interested in a simple way to store stuff? The digital and mobile revolution has touched us all, even those who have little to do with the IT industry. The majority of the world’s population now has access to a mobile phone and Statistica1 reports the estimation of over two billion smartphones in use this year. An exabyte of storage only represents 500MB of data for each of those users, or about 200 stored photos each. It’s easy enough to establish the need for exabytes of storage that’s available from anywhere; now let’s have a closer look at how S3 has been leveraged to meet that need.

S3 and AWS

The first and simplest use of S3 is for serving static web content. Because of the cleverly designed model of using buckets as part of the hostname, an entire website can be created using S3 alone. By using DNS mapping, available at any domain hosting company, a vanity domain like “www.myverycooldomain.net” is redirected to the S3 bucket, the default first page, such as index.html, and the infamous 404 page are configured and a static web site is born. Gone are the concerns about scale and availability, this website will support massive loads, and it has a great reputation for staying online. AWS only asks that you let them know if your load is going to change drastically overnight. Any alternative must support billions of objects in a bucket and massive numbers of requests — the vendor providing such technology better know what scale is all about! As usage grew, AWS added bucket-level access control (ACLs) to the original object-level ACLs, the first of many feature enhancements.

If web sites were the only way to distribute the world’s digital content, a few petabytes would have sufficed, but the Web 2.0 world arrived and user generated content totally changed things. Today’s web has billions of users sharing with billions of users and the quantity of data stored has changed by several orders of magnitude. Here is the second area of massive use for AWS S3: a repository for user data. As simple as S3 may be, it’s not a protocol for the masses, and application interfaces that allow people to store and share digital content have been one of the most dynamic areas of growth in recent years. Services like Dropbox, Pinterest and Reddit to name a few, have leveraged AWS to provide a scalable platform for collaboration. The ability of an HTTP REST interface to provide the ubiquitous universal resource locator (URL) makes so much of this shared data storage powerful: now any stored content is only a HTTP GET request away. In response, AWS added more features to S3, notably the ability to upload and store very large objects using multi-part upload, notifications to take an action when data is added to a bucket, and versioning since humans have a well-known propensity for changing their minds and making mistakes.

IT infrastructures

One of the fascinating outcomes of the S3 model has been the ability for new companies to concentrate their resources on perfecting application ergonomics and business models rather than on architecting a service platform. The corporate world now longs for IT infrastructures that permit this same level of elasticity and allow their business to concentrate on their business. While consumers have generated huge volumes of data, industries have also seen massive growth in the data they must manage. The “big data” revolution has also clearly demonstrated that enterprise data stores can contain highly valuable information when exploited and studied appropriately. As Pat Helland of Salesforce.com has said “accountants don’t use erasers.” Data is increasingly being kept indefinitely as a source of history and knowledge in the corporate world.

The S3 protocol is a popular choice for application development and corporate developers would gladly use the protocol, if only they could store the data on site and in conditions acceptable to their business. This is an area where an S3-compatible and scalable solution has its strongest appeal. The enterprise world has certainly placed strain on Amazon’s simple data model, with expectations coming from corporate hierarchies, security departments, and the rich protection models in place for many years now with Active Directory and LDAP user management platforms. AWS has responded with IAM, its federated identity management tools and server side encryption. These mechanisms allow the granular rights management familiar in the corporate world as well as the ability to allow now AWS users to interact using their corporate ID transparently on the platform. Finally, the server side encryption functionality allows corporates to have a higher level of security associated with their data. Many will require dedicated private platforms for data governance, but SSE is a strong step in the direction of full data protection.

With this increasingly rich functionality, AWS S3 isn’t quite as simple as it once was, but has become more useful and is becoming indispensable as the persistence component in on-demand infrastructures. Scality was early to market with S3 compatibility in 2011, and has made a strategic decision to apply our industry-leading knowledge of scalable storage infrastructures to providing the best on premises variant of the excitingly dynamic S3 protocol to those who cannot or chose not to deploy their infrastructures on a public cloud.

In our next post we’ll explain how Scality is able to follow the fast moving feature deployment of AWS S3 interface, and how we are making its deployment simple for non-cloud experts as well as making the interface readily available to the developer community.

1https://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/

Read the Press Release for the Scality S3 Server

Discover the Scality S3 Server

Follow Brad King on Twitter

news & events

New NAS Archiver

Tier inactive data off from primary NAS storage to Scality RING

In the News

The redesigned and improved MetalK8s 2.X

Events Updates

What the Corona Virus means for Events in 2020

 

Get a Live Demo

Request a demonstration

 

SAN FRANCISCO, USA

149 New Montgomery Street, Suite 607
San Francisco, CA, 94105
Email: sales.us@scality.com
Telephone: +1 (650) 356-8500
Fax: +1 (650) 356-8501
Toll Free: +1 (855) 722-5489


PARIS, FRANCE

11 rue Tronchet
75008 Paris, France
Email: sales.eu@scality.com
Telephone: +33 1 78 09 82 70


WASHINGTON, D.C., USA

43777 Central Station Drive, Suite 410
Ashburn, VA 20147, USA
Email: sales.us@scality.com
Toll Free: +1 (855) 722-5489

TOKYO, JAPAN

Otemachi Bldg. 4F, 1-6-1, Otemachi
Chiyoda-ku Tokyo, 100-0004 Japan
Email: sales.japan@scality.com
Telephone: +81-3-4405-5400


LONDON, UNITED KINGDOM

20 St Dunstans Hill
London, United Kingdom, EC3R 8HL
Email: sales.eu@scality.com
Telephone: +44 203 795 2434

Products

The most powerful data storage platform.

Protect, search and manage your data on any cloud.

RING

ON PREMISES, PRIVATE CLOUD STORAGE SOLUTION

ZENKO

DATA MANAGEMENT ACROSS
MULTIPLE CLOUDS

NAS ARCHIVER

TIER INACTIVE DATA FROM NAS TO SCALITY RING

Hardware Alliances

Delivering fully integrated solutions.

Learn More

Read about Scality data storage and management solutions.