The Future of Storage
The disruptive transition to software-defined
Sizing the opportunity
November 3rd, 2014
By Jérôme Lecat, CEO of Scality
I. Executive Summary
Software is eating the world
Marc Andreessen famously stated, “Software is eating the world.”1 Beyond the strong prophecy, it is interesting to look at what it really means for the IT industry, and specifically, for the $100 billion storage industry. Many of us have gone through several technology waves, and the pattern repeats itself time after time, in what is known as the S-curve.2 When a new technology disruption is introduced, the observers are divided between the “bullish,” who are overly optimistic about the speed of change, and the “doubters,” who believe that the new technology will never replace the status quo. Indeed, it is extremely difficult to state at the introduction of a new technology whether it will succeed, but when it does, its start is slow, but ultimately its supremacy is often beyond the “bullish” expectations.
At Scality, we believe that software will eat the storage industry; we call it Software Defined Storage (SDS).
The disruption that will happen to the IT storage industry is similar to the disruption brought onto telecom by the Internet or to retail by Amazon: a complete change in the value chain. As a result, traditional vendors will lose billions of dollars in business while new ways of doing things will attract billions of dollars in business opportunity. Software running on commodity hardware is the prime enabler of this disruption. Scality embodies this transformation, with field-proven software that operates on any commodity server, without any certification program or other hardware-attached attributes. Scality’s software, the RING, has been designed from scratch to bring to large enterprises the solutions used by Internet giants.
This study aims to quantify a realistic estimate of what the disruption looks like in the next 6 years, with a 2020 horizon. We will especially focus on scoping the addressable market for Scality: the software opportunity in capacity-driven Software Defined Storage.
By 2020, the storage industry will be dramatically different:
Less than 25 percent of storage spending will be in small and medium size datacenters
Ten years from now, a limited number of small enterprises will operate their own IT. They will leverage cloud and SaaS services such as Salesforce, Office 365, Box or Amazon’s AWS. In its Datacenter Forecast,3 Gartner reports that in 2010, 42 percent of datacenter spending on x86 servers and storage systems was for datacenters with fewer than 100 racks, and by 2018, this number will have fallen to less than 30 percent. We expect this trend to continue so that by 2020, less than 25 percent of storage spending will be for smaller datacenters. In essence, 17 percent of the market will disappear for traditional vendors. Obviously, there will be a storage opportunity for the cloud and SaaS providers, but it will be of a different nature.
40 percent of storage capacity will be sourced directly from disk and component manufacturers
Historically, established storage vendors like EMC or NetApp have enjoyed gross margins of about 55 percent; an impressive level when compared to other players. Hard disk drive (HDD) manufacturers (Seagate, Western Digital) typically achieve 30 percent margins, while original equipment manufacturer (OEM) server brands (like HP and Dell) are below 30 percent, and Taiwanese server manufacturers (like Quanta) are below 10 percent. Much of EMC and NetApp margins come from the high-price resale of components, including HDDs. The availability of Software Defined Storage will put tremendous pressure on such margins, as it becomes easy for large buyers to procure the software and commodity server separately. Furthermore, since reliability is built in the software, not the hardware, customers are incented to buy lower quality servers. As a consequence, we expect original design manufacturers (ODMs), both server ODM and HDD manufacturers, to sell systems directly to the larger datacenters. Already today, some large public cloud providers and other large storage users procure their HDDs directly from the manufacturers (mostly Seagate and Western Digital/HGST). According to IDC, this business grew from a mere $0.4 billion in 2010 to $2.1 billion in 2014, and will grow to $3.1 billion by 2017. We believe that this phenomenon will accelerate, enabled by the emergence of SDS software. Investment bank J.P. Morgan estimates that 30 to 35 percent of the server market will shift to ODM in the next 5 years.4
Most large enterprise and service providers will continue to operate their own IT, but will do so with a software-led approach
While small and medium size businesses will leverage cloud services; the service providers, large enterprises and many vertically focused players will continue operating their own IT, but they will do so in a way similar to Google, Amazon and Facebook. This group represents about 20,000 entities worldwide that will be leveraging standard servers with software providing both the application and the infrastructure intelligence. By 2020, the transition will have already begun, and about 50 percent of these large enterprise and service providers will already have transformed most of their IT operations.
Scality addresses 80 percent of storage workloads, representing one of two storage tiers in the software led future:
Analysts often assess Scality as part of the object storage market, which is usually defined as both the internal organization of data on HDDs and as the interface to the application. Defined as such, the object storage market is seeing moderate growth, because only new applications (e.g. mobile applications and websites) have adopted object interfaces to date. Most enterprise applications rely on traditional file and block interface methods.
The Scality RING is architected around an object storage core technology, but it presents traditional interfaces to applications. As a consequence Scality regularly competes with or replaces traditional SAN, NAS and scale-out NAS systems. Indeed over 50 percent of Scality’s customers use RING as a scale-out NAS. Scality addresses a market that is much wider than analyst-defined “object storage market.”
We believe that in the software-led world, the storage infrastructure will be collapsed to two tiers.
For applications that are latency sensitive, like ERP, relational databases, VDI, or high-frequency trading, the storage will be provided in silicon. Several architectures are competing, including all-flash arrays like Pure Storage, VM-optimized hybrid-storage like Tintri or converged systems like Nutanix.
For all other applications, including SaaS, user generated content, documents, content and video streaming, large data repository, big data, distributed computing and long term active archive – what we call capacity-driven storage – the preferred method of storage will be a scale-out, object-based software, like Scality’s RING, deployed on commodity servers.
The idea of storage moving to a two-tier architecture, and latency (not IOPS) being the key differentiator, is not new. As a point of reference, Richard McDougall, from the office of the CTO of VMware, explores this idea in an article titled, “Is Cloud Storage going to disrupt Traditional Storage? – Part 1: The demise of expensive datacenter storage”5 in 2010! Cloud storage is only now starting to be a reality profoundly affecting market dynamics.
Altogether, tens of billions of dollars of value will shift in the new storage landscape:
- The storage industry as a whole will not grow beyond $100 billion, as new cheaper storage replaces more expensive traditional options
- The traditional storage systems market will shrink from $38 billion to $20 billion in six years (10 percent negative CAGR)
- The opportunity for software defined capacity-driven storage will be $42 billion, of which the market opportunity for Scality6 is $5 billion
This represents a huge change for the IT industry and its vendors. The mid-range of the market is essentially going to disappear, while the high-end will be completely transformed.
IDC talks about the IT industry “transitioning to the 3rd platform,” while Gartner calls this transformation the “Web-scale singularity” and states that “during the past decade, leading cloud services organizations have been experimenting with new ways to deliver IT services. Their efforts have resulted in a singularity event that changed the trajectory of the IT landscape. What is being left behind is IT conventional wisdom”7 Wikibon, an analyst firm specialized on the storage market, calls this transition the “Server SAN” and writes “Server SAN is software-led storage built on commodity servers with directly attached storage (DAS). Wikibon believes Server SAN is poised to disrupt traditional storage architectures over the next decade. As enterprise organizations begin to replicate the infrastructure of hyperscale giants, software will lead the transition.”8 A recent article from The Register titled “SDI (Software Defined Infrastructure) wars: EMC must FORGET ARRAYS, adapt or disappear9”discusses the impact of such drastic change on the established market leader.
When asked how big is the market opportunity for the software that will be at the heart of future storage infrastructure, ESG founder, Steve Duplessie responded: “You would need a crystal ball to answer this question. It is going to be a completely new order, but I’d say that with a $5 billion estimate you are very conservative.”
The body of this paper will present the four detailed approaches we used to size the disruption of the storage industry:
- Market Size per IDC File and Object Based Storage Forecast
- Market Size per Scality’s Analysis of Industry Trending
- Bottom-up Approach from Vendor Revenues
- Bottom-up Approach from the Number of Potential Customers