Dailymotion serves 300 million monthly users with Scality
Dailymotion is the second largest video website just behind YouTube. We get about 300 million unique visitors and deliver three billion streams per month.
As of June, 2015, the storage infrastructure is about 35 petabytes. We have about 50 million videos in stock, each one in three through seven qualities. And we see a 35 percent growth rate per year. We have to add about five petabytes per year. So I guess 25 to 40 petabytes in five years. Our cluster is deployed on 61 HP servers, and that’s an impressive 3,660 hard drives spinning all the time in the data center. Dailymotion delivers video worldwide from four main locations. And we also use external CDN vendors like Akamai, EdgeCast, or Limelight.
Our main storage challenges are TCO, failure resiliency, and ease-of-use. It’s really important that we can choose the hardware that gives us the best density. As of today, it is HP. And we also like the fact that this kind of software is running on Linux, because it’s our operating system of choice for the servers we operate. It means we have a 25% overhead to implement that data protection scheme, but it’s quite low compared to a pure replication system thanks to Scality using erasure coding in its software. We didn’t to have to change anything to our application to use the Scality REST protocols because the application was already designed around the abstraction of document storage.
The main technical benefit we experience from implementing Scality RING was the ease-of-use on off-the-shelf hardware. The implementation of Scality was completely transparent to the end users. There were near to no issues within the implementation of the project, because everything worked as advertised. Scality helped us achieved our project goals by reducing our TCO.