Nonstop exponential data storage growth is a well-known fact of modern life. The challenge is far worse with unstructured data than with structured data. IDC forecasts 3 times the growth rate of unstructured as compared to structured data. This is particularly acute for Web 2.0 organizations, where just a small change in the number of users can cause stored, unstructured data (user photos, blogs, and videos) to expand by multiple orders of magnitude.
Solving Web 2.0 storage growth and integration presents challenges of:scale, transparency, performance, integration, uptime, and much, much more
Key Web 2.0 Market Challenges with Traditional Storage.
Storage technology that was designed before the Web 2.0 HTTP interface was standardized makes for very inefficient integration. A Web 2.0 service can never be down or its reputation will be seriously damaged, and its users will look for other social networks they can rely on. This presents a number of challenges that traditional storage systems are simply unable to meet in a fully satisfactory fashion:
- Inability to scale storage systems to levels required by Web 2.0
Traditional systems lead, inevitably, to storage system sprawl, with excessive expenditures of time and money on management, infrastructure, and ongoing data migrations—all in an ultimately futile effort to chase down the root causes of data storage and retrieval problems.
- Inflexibility of architecture designed for legacy block or file interfaces
Modification of traditional storage systems to support the native web development framework requires extensive knowledge, extra work, and workarounds that are often inflexible.
- Inability to support the 7x24x365 always-up, never down Web 2.0 market
SLAs are not a measure of hardware reliability, but rather a measure of Web 2.0 service availability.
- Excessively high TCO
Total cost of ownership is based on a model that does not work for Web 2.0.
Traditional or legacy storage does not make life very easy for the Web 2.0 storage administrator. It tends to be overly complex, making it exceedingly difficult to expand object or file storage capacity to the levels commonly required for Web 2.0 services, and to maintain system performance at the same time.
Traditional storage is also incapable of remaining online without incurring incredibly high costs to compensate for scheduled or unscheduled maintenance tasks that would otherwise interfere with data availability During a tech refresh, data migration alone severely impacts storage uptime and availability. In corner cases where traditional storage can meet the challenges of scaling, performance, and availability, it does so at such an exceedingly high cost that it becomes financially untenable for the Web 2.0 service provider business.
Essential Web 2.0 Requirements
The high-paced competitive nature of the Web 2.0 market makes it imperative that service providers utilize storage that meets or exceeds the following requirements or risk falling by the wayside:
- Ability to scale to billions of objects while maintaining performance transparently for all users without disruption
Web 2.0 storage must be able to accommodate PBs to EBs of capacity, and billions of objects or files, with satisfactory performance for millions of concurrent users.
- Storage that fits the native web development framework
Support for the REST and HTTP protocols.
- Always available and online
Must provide a minimum of three nines (99.9%) availability all the time.
- Lowest possible TCO
Increasing numbers of Web 2.0 providers place very high market pressure on cost, so any storage system must closely match these revenue models.
Storage is one of, if not, the largest Web 2.0 cost components, and it has an outsized effect on the quality of services offered. It doesn’t take very many users to quickly stress a Web 2.0 storage infrastructure. Just a few thousand users will easily and rapidly generate petabytes of stored data and billions of files. Adding user and market demanded capabilities to the Web 2.0 services is best addressed with the simple Web standardized REST interface. Otherwise, time-to-market increases, competitiveness suffers, growth declines, and costs increase.
Perhaps the most important user demand is availability. Users tend to be unforgiving about downtime. They require the Web 2.0 service as well as their data to always be available no matter what the time of day. When it’s not available, they start looking elsewhere.
Web 2.0 storage must therefore be flexible and always online, providing for transparent tech refreshes, and enabling all scheduled and unscheduled maintenance tasks to be performed without any user disruptions—at the lowest possible cost. It must also more effectively align its costs with the Web 2.0 revenue model, whether that model is based on advertising, click-throughs, or subscriptions. Traditional storage systems charge everything upfront, putting all the risk on the Web 2.0 service provider. A storage system pay-as-you-go model is a much better fit.
The Solution: Scality RING™ Organic Storage
Scality RING Organic Storage is architected from the ground up to meet and exceed all Web 2.0 service requirements. The RING storage interface is standardized for the Web 2.0 REST API (HTTP puts, gets, etc.). It scales capacity into the exabytes, files or objects into the billions, and provides superior performance to tens of thousands of users. The scalability of the RING solution is the direct result of its unique Distributed Hash Table (DHT). DHT is an extraordinarily efficient lookup methodology that enables retrieval of files or objects from very large numbers of storage nodes with an exceedingly high level of performance.
Scality Organic Ring Storage Solution Diagram
Scality RING Organic Storage provides unparalleled data, nodal, and system availability by leveraging its distinctive industry-hardened, carrier-grade service availability and data reliability. The RING also comes with unequalled built-in system data resilience similar to an organic immune system. Every node constantly monitors a limited number of its peers, automatically rebalancing replicas and load to make the system fully self-healing without human intervention. Consistent hashing guarantees that only a small subset of keys is ever affected by a node failure or removal.
The RING also rebalances the data load automatically when a node fails, is removed or upgraded, or when new nodes are added. RING makes technology refresh a simple, online process with no application disruptions, eliminating data migration, long nights, and sleepless weekends. The result is a very high level of fault tolerance because the system stays reliable even with nodes joining or leaving the ring.
Just as importantly, costs are kept low by using standard off-the-shelf commodity server nodes, and Scality delivers the storage on a pay-as-you-go (pay-per-use) model that more closely aligns with Web 2.0 service provider revenue models. Unlike traditional storage, the Web 2.0 service provider pays on a used capacity basis, not on raw storage capacity, thereby benefiting from the lowest possible storage TCO.
ABOUT SCALITY
Scality is the developer of RING, a software platform enabling cloud storage to easily scale up to exabytes using commodity server hardware with direct attached storage. Scality delivers the performance and reliability of a SAN- or NAS-based architecture without the aggravations of volume management—at one third to one half the cost.
Business uses for Scality RING include Web 2.0, webmail (storage for millions of users), Storage-as-a-Service, and other cloud computing service applications. Deployments manage billion of files while meeting or exceeding the high performance expectations of application users. Scality RING is based on a patented object storage technology that delivers high availability, ease of operations, and total data control.
PARTNER FOR SUCCESS
Partnering with Scality means gaining access to Scality’s extensive storage and Web 2.0 expertise, innovative technologies, pay-as-you-go pricing, intuitive integrated billing, training, business planning, best practices, branding, business tools, and even lead generation. Best of all, Web 2.0 service providers are able to continue using the applications and tools at their disposal, enabling and facilitating uninterrupted market growth. This permits investment to match revenue models, providing an immediate ROI, without the limitations of traditional storage solutions.
© 2012 Scality. All rights reserved. Specifications are subject to change without notice. Scality, the Scality logo, Organic Storage, RING, RING Organic Storage, are trademarks or registered trademarks of Scality. in the United States and/or other countries.