Scality's blog

Is “software-defined” just a new way to whitewash old products?

This is a post written by Scality’s CEO, Jerome Lecat, originally published on Venturebeat on December 5th, 2012

In the past year, we’ve seen technology companies offering software-defined networks, software-defined storage, software-defined servers, even software-defined hardware. It’s fast becoming a buzzword. But does it have real meaning?

In a recent tweet, Randy Bias, chief technical officer of Cloudscaling, stated, “‘Software-defined’ is the new ‘cloud.’ Sprinkle it on your products along with an API and you *are* the future.” Through this statement, ironic as it may be, Randy is posing some good questions: Are we seeing a new “washing” phenomenon like “cloud-washing” or “green-washing” of products? Is the “software-defined” concept even useful?

It is incredible what a $1.26 billion acquisition can do! In just the few months since VMware shelled out that total for Nicira, a maker of software-defined networking technology, I cannot count the number of startups which have rebranded themselves as “software-defined something.”

When it is a software start-up like Virsto, Datacore, Nexenta, or Scality (my own company), I can understand. But hardware startups? You have to be bold to call yourself “software-defined”! I will not mention names to not embarrass anybody, but there is undoubtedly some “washing” going on.

Beyond this, is the concept useful ? I believe it is.

The IT industry is regularly traversed by new concepts. Some are useful and they stick, some are not and they vanish. Some reappear a few years later under a different name. The Internet was a valid concept, and it stayed. But at the same time as the commercial Internet was developing, many established companies repainted their offerings as “extranets,” but the concept did not live long. VOIP (voice over Internet Protocol) is a valid concept, but it only appeals to technologists. At the end of the day, a phone call is a phone call, and I don’t care what protocol my phone speaks, as long as I can make calls.

I am still struggling with the “cloud” concepts. To be useful a concept needs to have a clear definition that is understood by everyone using the concept. “Cloud” actually has a definition, but the word is mostly used to mean something else than the definition. The first definition I have found is from Forrester Research and dates back to March, 2008: “A standardized IT capability (services, software, or infrastructure) delivered via Internet technologies in a pay-per-use, self-service way”.

With such a definition, the concept of “private cloud” is a pure contradiction. But the use of the term has evolved, and when people say they “store their documents in the cloud,” they really mean nothing different than what they meant three years ago when saying that “they pulled a document from the Internet.” I actually often joke that when I was selling Internet access in the late nineties, I would draw a screen (that was my customer), a line, and a cloud (which I labeled “Internet”). Some fifteen years afterwards, I do the same diagram, but now I have replaced the word “Internet” with the word “cloud”!

But “software-defined” is different. I can easily picture a world in which networking and storage are delivered by software running on standard servers and how this world contrasts with today’s datacenter, where networking and storage are delivered by special-purpose appliances.

Actually, when Scality came out of stealth in 2010, we were struggling to position ourselves. I would explain that we are a “storage software” company, and people would look at me with a grin. “Storage software? So what do you use for storage? A storage array from EMC or Netapp?” I would answer patiently that you don’t need a storage array, that we transform a set of generic servers into a storage platform. Most of the time I would be met with a more inquisitive look. In the defense of the puzzled people I was talking to, every analyst research firm out there considers storage as being part of companies’ hardware spending, so it is very difficult to conceive of storage as being software.

But in the last few months, I have been trying out the term “software-defined storage,” and people immediately get what we do. This in my mind is the sign of a useful concept.

Jerome Lecat

“Great Design is key”, they said – well, not the Enterprise CIOs.


San Francisco – November 9th, 2012

VentureBeat’s Christina Farr wrote and excellent article about the Myth called the “Dropbox Effect”, which is the assumption that a well-designed and easy to deploy product, once successful with consumers, and hence employees of enterprises, will prompt CIOs to fork out a large pile of money to integrate it officially as part of their internal IT.

The main issue with this is that it doesn’t work, and that is very well detailed in the article and also by Ben Horowitz in the linked blog post on the same topic.

Design is one thing, and it surely can explain the success of products/services like Dropbox, Box[.net], Instagram, Asana, etc but just as important when creating a new solution is to clearly define the target or targets. This should influence the way the solution is packaged, designed and monetized. And one size fits all is hard to conceive, let alone implement.

Maybe multiple solutions,one for each target is the way to go, if one has enough resources and funding to afford it. Look at Dropbox, they’re insanely popular with consumers, but they are clearly struggling to penetrate the enterprise as an official IT application (as opposed to a renegade app employees use on their own, until IT blocks it – which is something that goes against Dropbox).

The economics work if you design your system with a specific target and don’t move away from it. In this video, Phil Libin from Evernote explains the model they used when they designed the product, and how they kept to it without trying to “pivot” into a new segment or model. Now they’re very profitable, and are enjoying a very good reputation with a great product, perfectly designed for its market.

Again, great design is obviously very attractive, especially if you’re going after the consumer market, but it might not be what the enterprise users are going to look after. CIOs will look for other features (reporting, security, ACLs, provisioning integration..). If a product has both, then that’s the holy grail.

Scality is sponsoring VentureBeat’s CloudBeat on November 28-29, in Redwood City, CA, and this is the kind of topics that will be addressed during the keynotes, and on the floor.

See you there.

-Marc

The Object Storage story doesn’t end with Archive.


San Francisco – November 8th, 2012

In the last few weeks, there has been multiple panels about Object Storage in different trade shows around the world, which is great. However, I disagree quite strongly on the content delivered by the panelists, in that they were limiting Object Storage to a very specific use case. I know for a fact that Object Storage is much more than merely a technology for Active Archive.

My friend Tom Leyden, of Amplidata, with whom I’ve had many discussions about this, wrote a post summing up all these panels, and there are some interesting points that were raised by the crowd with regards to Object Storage beyond the points made by the panel. It is worth a read.

It’s great to see that those panels were in general well-attended. It shows Object storage is getting some attention. It still has some stigma and a bad rep because of the first generation of object storage, created in the late 90s.

Since then, the concept has been rethought and a new generation of Object Stores have come up and solved the issues the first generation was not able to cope with, mainly offering true massive scale, better APIs, higher performance and reliability, and also and maybe most importantly, cost-efficiency. This was very well explained in the panel. However, Object Storage for Large-Scale Active Archive storage to replace tape and “monetize one’s archive” was the main use case put forward during the panel. There are two things here that I strongly disagree with.

First, I don’t think Tape is dead as it was sort of implied sometimes during the panels (and not only by the panelists). Yes, it has strong limitations, but there is to this day no other technology that can manage low-consumption for hundreds of PBs of data. The very low-latency and inconvenience of Tape is the price to pay to be able to store this amount of data fairly reliably and cost-effectively. Sure, there are some technologies like MAID, and others that are trying to fix this, but the low-consumption factor is going to be hard to challenge, even with these techs. After all, one of the most innovative companies in the world (AWS) has decided to use Tape for their Glacier offering (very low-latency storage for a very very low price / GB).

Second, Object Storage is not limited to Archive. It is obviously one of the most common usage as it’s a very cost-effective way to store huge amounts of data, and still have them available quite fast. But it’s also a legacy of the older generation. At Scality, our vision from the get-go was that distributed storage should be able to deliver high-performance primary storage for applications. And to make things clear, when I say high-performance, i mean low latency, high throughput. And we can also manage high IOPS.

What Object Storage cannot really manage very well today is high performance on Random I/O, which is what’s needed for VM Storage, Database storage etc. For more sequential I/O patterns, it can be done with an Object Store like Scality. Moreover, by adding native automated tiering and by being hardware-agnostic, Scality can manage High-performance tiers, a second capacity-based tier and even could handle Tape as a third long-term capacity tier. That’s what we think Object Storage should be all about: making storage easy for the applications.

The Valet analogy is great, but I think the Valet should be able to follow some instructions like “Park my car close, i’m gonna need it soon”, “you can park it far away, i’m gone for 3 days” etc.

All in all, I think this shows that we are still struggling to get away from the stigma associated with Object Storage. The Object Storage Industry is seeing a lot of innovations these days, and obviously not only by Scality. And this is being recognized too slowly by the industry, the Analysts and the Press. That’s why we’re strongly behind the Next Generation Object Storage Summit, which will happen at the end of November in Miami, organized by Steve Duplessie (who was by the way in one of the panels) and his ExecEvent. It is a similar event than the one he created for Tape (The Tape Summit), even though the motivation behind it is fairly different.

The summit is really about gathering the main actors of the new generation of Object Storage industry to come up, together, with a strong message about our the advantages of our Innovations in the field. The goal is to make sure that Object Storage is not being overlooked anymore as an old technology with many flaws and caveats to really be considered for massive scale storage by technologists whether they work in Enterprise IT departments, Telcos, or Service Providers.

I’d strongly urge you to come and talk to us at the summit, at the many shows we spend time at to evangelize our industry or directly on our website. There’s so much more about Object Storage. We’re dying to let you know ;)

-Marc

The Container Shipping industry schools Storage about Efficiency


San Francisco – October 26th, 2012

As Internet use continues to skyrocket, the volume of files being produced, shared and stored climbs at a tremendous pace.

The traditional methods of storing this kind of ‘unstructured data’ show signs of failing under the strain of this completely unanticipated load. In reaction, system administrators work furiously to plug the fissures appearing in their infrastructure, and CIOs cast about urgently trying to assess and source potential solutions to these tectonic shifts.

Although this is a new and unique challenge within the data storage world, other industries have been through equivalent shifts in scale, and the solutions that they evolved to might serve as illuminating parallels for these beleaguered decision makers.

Copyright Fred R. Conrad/The New York TimesCopyright Fred R. Conrad/The New York Times

The shipping container industry might seem an unlikely source of relevant information but holds some poignant insights on scalability, flexibility, automation and cost efficiency.

The story of this parallel system is as old as civilization itself. Trade was based on the slow, arduous and often unpredictable transport of relatively small quantities of high value items, along established trade routes. But as populations grew and technology evolved, the traditional demand and supply chains broke and reformed and the markets for manufactured goods exploded.

It took quite some time for the transportation industry to respond to this radical shift in scale, but in 1953 container shipping was invented and the shipping industry entered a new era. Instead of transporting individual bags and boxes on ships and trains and camel caravans – packing and unpacking them at each stage of their journey – the container consolidated vast quantities of goods and transported them en masse aboard massive ships to centralized transportation hubs where whole containers could be picked up and forwarded by train or truck, onward to their final destinations.

Probably the most obvious lesson to be derived from this history is that the scale of everything had to change radically. The size of the ‘boxes’, the size of the transporting vessels, the technology necessary to move these enormous containers on and off ships, trains, and trucks.

Of course, where there are such massive changes in the scale of a system’s operations, it requires equivalent changes in the way those systems are automated and administered. Indeed if you are trying to get several thousand containers onto and off a single vessel simultaneously and manage dozens of such moves a day, you are going to need some serious planning, precision scheduling and some very intelligent automation systems.

Today container shipping is a highly automated industry. Computers determining the optimal placement of boxes, to keep the contents safe and ensure that loading and unloading is expedited. Computers schedule the arrival and departure of trucks, trains, ships, and computers chart courses and more or less run the container ships – with not much more than a skeleton crew.

The whole operation needs to run with complete predictability and split second timing because at these scales – damaged or lost containers (or ships) or seemingly minor delays in unloading cost millions, even billions, of dollars in lost revenue, spoiled goods, and angry customers. All of this leaves very little room for human error.

Probably the most important lessons to be derived from the container shipping industry though – relate to the commoditization & standardization of the components and systems. The interchangeable nature of the containers, the ships, the cranes, the trucks, and the trains keeps the infrastructure and management costs to a minimum, and ensures complete flexibility in the event that one of these components needs to be ‘swapped out’, and allows for the smooth transition of goods from source to destination.

This aspect is an integral and fundamental aspect of the industry’s success. It is the interchangeable nature of the components that makes the system work.

So in summary the container shipping industry teaches us to take scale very seriously – choosing an infrastructure that scales, linearly, without introducing new costs associated with that growth. When we are applying this to Storage, these lessons must be applied to both capacity and performance, adding a dimension to the scale. So we need to select infrastructure that can natively scale the number and size of files, and the speed at which they are accessed and moved – by design, and not as an afterthought.

Automation is not optional. At scale, human beings become a liability – they operate in too serial a manner to keep up with the parallel demands and split second timing needed by systems like these. The systems themselves need to be intelligent, not just automated but problem solving. Ideally they should be able to identify and work around problems. In our software derived, virtual world they can even heal data, rebalance loads and retire infrastructure.

Commodity hardware and standard interfaces are essential in rapid growth, low margin environments. There is simply no room for the costs associated with vendor lock in, and absolutely no time to be wasted on developing custom interfaces between individual applications, or appliances.

Scality RING delivers native scalability and high performance cost effectively with its Object Storage, matching all the characteristics we just mentioned. Its organic, self-healing architecture is highly automated, minimizing the risk of human error.

And it is completely hardware vendor agnostic – running on commodity x86 hardware – thereby ensuring that CAPEX costs are kept to a minimum, and that appropriate density hardware can be utilized as necessary. It also ensures that as hardware technology evolves, data can be easily and seamlessly moved to the newer hardware.

It also offers a range of standard interfaces including REST / HTTP, CDMI and a Scale Out File System.

Marc Villemade & Monique Shefer
Scality Team

Open Standards are the key to True Cloud. Not Open Source Stacks.

New York – October 12, 2012

I have written about this before (here and there), but it seems that there’s a massively important point that is being lost in the midst of all the discussions about Open Source Cloud Stacks being the only way to make the Cloud a true, hegemonical success. And that point is that it’s not.

The only way to enable Public, Private and Hybrid Clouds to collaborate efficiently so that users can truly take advantage of this new era of computing is Open Standards. Only when these have been designed and broadly adopted (and Open Source will massively help here) will we be able to call the Cloud a revolution.

And we’re obviously not the only ones pushing for Open Standards. Canonical is deeply involved in Open Source of course, but is a very strong advocate for Open Standards as well. They are after all one of the leading distribution in the Cloud (both as Host and Guest OS), have their own personal Cloud Storage Service (Ubuntu One) and strongly evangelize the Cloud Revolution.

They are also part of the OCI (Open Cloud Initiative), whose goal is to design and promote a complete standard for Cloud: protocols, guidelines, reference architectures so that designing, building, maintaining a Cloud and interoperating with others is possible. Today, it is not that simple, because none of those things have been clearly defined and/or standardized. Canonical is pushing for that, and so are we.

Scality’s approach when it comes to Storage Protocols is radically different from other Storage Vendors and in the same vein as Ubuntu’s. Instead of creating yet another proprietary interface, Scality has recently announced their Open Cloud Access (OCA) Strategy, at the core of which sits an abstraction layer that can be used to plug in any protocols. Today, this includes various REST protocols – Scality’s own documented REST APIs, RS2 (an Amazon S3 compatible API), CDMI – and even a file system mount. Scality intends to extend these access methods to OpenStack and NFS in the coming months. And we’ll be able to add more protocols as they come around and become standards. Again, opening up our storage systems to all protocols is our pledge to the Cloud

Scality will be in San Diego next week, at the OpenStack Summit, showing our strong belief in Open Standards. While we do recognize Open Source as a strength that will foster a faster adoption of these new Cloud technologies, we strongly believe that Open Standards is the key for Cloud to really become Ubiquitous.

I will be there, and so will Philippe Nicolas, Our director of Product Strategy, and Giorgio Regni, our CTO.

Please contact us if you want to meet at the Summit and talk about all this. It is a great conversation to have in these times.
And look out for this blog as well, as I will be posting some very cool news about Scality and about the Summit as well.

Looking forward to seeing you in San Diego

-Marc

Of Clouds, Storage and Openness

San Francisco, CA – Tuesday, September 18th

As I have previously talked about on this blog, there has been a significant amount of discussions about the Openness of the Cloud this year. In the League of Clouds, I was saying that it is necessary for the industry to start working on a set of standards used by every cloud Service or Software provider to ensure interoperability, which will in turn limit lock-in, boost adoption and in the end benefit all.

Analysts are saying that the way the industry is going, Hybrid Cloud, Hybrid IT and Cloud Bursting are in their growing phase. If they are to go through that phase and mature to be of significance, it will be necessary for a set of Standards to emerge.

While I agree that Open Source is necessary for a new technology or new paradigm to gain mainstream adoption and go beyond the Trough of Disillusionment and into the Slope of Enlightenment (as per Gartner’s Hype Cycle Taxonomy), I do think that Open Source will not appeal to everyone (for different reasons, and different markets/verticals). And in that case, establishing Open Standards becomes indispensable for interoperability between Open Source and Proprietary software so customers can benefit from the full potential of the Industry.  And, of course, Proprietary software can follow Open Standards, even though it is not Open Source.

Today, Scality announces their commitment to a true Open Cloud Access strategy for Data Storage. Yes, Scality is a proprietary solution, but with this statement today, we’re showing our trust in Open Standards. At the core of this release is a CDMI* Server implementation, which is the standard pushed by SNIA. It is our belief (and that of others in the industry) that CDMI represents the best solution available today, when it comes to independent standards.

However, Scality also recognizes that the Industry might not recognize CDMI’s access protocols as the standard, nor do we believe in having customers redeveloping their legacy applications to meet these new standards. That’s why earlier this year, we added to our already existing REST APIs and RS2 (an Amazon S3-compatible interface) interfaces, a file system mount. And in the coming months, Scality plans to extend its compatibility to other ways of accessing Data: CloudFiles, NFS, and more.

With this release, our customers now have access to the best software-defined storage available for unstructured data.  RING provides massively scalable, no hassle storage for their private environment, with the needed cloud interoperability  (whether for Amazon or Open source flavors implementing CDMI or CloudFiles) for implementation of Hybrid Cloud & Cloudbursting.

We strongly believe that this is where the next generation of storage, of which we are part, should be heading for. Object Storage vendors can be competitors but there should be a unified message around open standards accepted and followed, whatever the vendor’s specific features, or target market. This is actually one of the topics of the Object Storage Summit, organized by The Exec Event’s Greg Duplessie on November 28-29.

Open Standards are the key.

 

* CDMI: SNIA’s Cloud Data Management Interface.

Remembering Claus Egge – 1958-2012

New York, NY – Tuesday, August 28th

One of the most respected analyst in the Storage industry, Claus Egge, has unfortunately and very suddenly passed away late July. I didn’t know him that well personally, but I had met him at shows and talked to him over the phone a few times. He was a very smart person with an incredibly humbling knowledge about Storage and a kind soul. And from all the testimonies on the internet about his passing, he truly was ready to help you whether professionally or personally, always offering to show off his country and city to visitors, for example.

He will be remembered as a very smart, talented and professional individual who knew the Storage Industry inside out and overall as a gentle, modest and very witty person. It is very sad that it is always when things like this happen that we regret not knowing someone better. I do wish i had.

For those who want to pay their final respect, you can do it here and there will also be a ceremony held at the Duchess of Cambridge, 320 Goldhawk Road, London W6 0XF, from 6pm onwards on Wednesday, September 5th. Please RSVP.

Also, a fund has been set up at the Heart Foundation to support much needed research.

The Scality Team and myself are thinking about his family and friends in these hard times.

Rest in Peace, Claus.

-Scality’s team

Let’s climb the Object Storage Mountain

New York, NY – July 30th 2012

A couple of weeks ago, I attended the Big Event, organized by Greg Duplessie in Santa Clara. The 2-day event gathers 40-50 executives of the Storage Industry from startups like Scality to big vendors like EMC.

There were a lot of interesting sessions, ranging from an analysis of the current VC market, to a discussion about Mergers and Acquisitions to an open discussion on Marketing 101 for startups.

But of all of those sessions, I felt like one of the most interesting sessions was the one, instigated i believe by one of the vendors (Quantum, not to name it), about an Object Storage Summit. The idea was to gather Object Storage Vendors as well as the ecosystem around the technology and have an open discussion on whether or not it’d make sense to have a similar thing as the Tape Summit for our industry.



 [Read more...]


		

		
	

		


				
The League of Clouds
 
		
				

			












San Francisco – July 13, 2012


There has been a large deal of discussions about Cloud Federation in the past year(s), led by a lot of prominent bloggers (amongst which @krishnan who has been pretty relentless, in a good way, about it). I truly believe it’s the only way Cloud is going to be able to grow up from teenhood.


Only a few weeks ago, some pretty vivid and colorful conversations about Federation, Open Source, Standards and API Cloning happened at GIgaOM’s Structure in San Francisco between Rackspace’s Openstack, Eucalyptus, Citrix’ CloudStack, AWS… The usual suspects.




 [Read more...]


		

		
	

		


				
True Cost of Storage is a hot topic !
 
		
				

			











July 3rd, 2012


George Crump from Storage Switzerland, back in January, wrote an article about the True Cost of Storage that clearly stated how to really calculate one’s Total Cost of Ownership (TCO) when it comes to Storage.


Through all our discussions about storage with prospects and customers, we realized that the real, hard cost of storage was not always well calculated, and pretty elusive to the IT managers. It was made even more obvious when comparing to the new types of Storage available to customers, especially Cloud Storage since the models, both economic and technical, were quite different.




 [Read more...]


		

		
	

	
Next Page»