OpenStack Vancouver Summit Technical Wrap-Up

The OpenStack Summit in Vancouver ended a few days ago. I was fortunate enough to be there with several colleagues from Scality. There were many interesting sessions and I’ve captured the high points below.

A look at the new OpenStack projects governance

As Thierry Carrez, OpenStack release manager, puts it in the description of one of his talk at the summit:

OpenStack was previously defined by one thing: “the integrated release.” This single term ended up meaning different things for everyone. It both became too large and too small an artifact to represent what the community does. It was too big for many people to deploy, and too small to fit the features that many people wanted. It was time for a change.

Starting from now, there will be no more distinction between “integrated” and “incubated” OpenStack projects. Any project could be considered as OpenStack as long as it follows the OpenStack Mission (to provide Infrastructure as a Service [IaaS]- or build on top of IaaS), it is open (in terms of licensing, community, development, etc.), it interoperates with other components (e.g Keystone for authentication), and submits to the Technical Committee oversight.

The direct consequence is the multiplication of OpenStack projects. To keep a steady fast development pace, projects must be fully autonomous. Practically, the development environment installer (code named DevStack), and the upgrade path tester (code named Grenade) should be fully modular and accept plugins. The testing framework (a.k.a Tempest) will reduce its scope to integration tests only, leaving each project in charge of its functional testing. The API documentation will be moved from a central place back to each project.

Finally, the release model will change; there will be no “integrated release” OpenStack Liberty. Instead, there will be an “opt-in coordinated release,” 6 months from now, with the Release Team providing milestones, process. and tools. As a side note, projects are now free to be released more frequently than once every 6 months.

From a technical point of view, this new governance model (internally referred to “the Big Tent”) looks good, but the communication and the marketing aspects around OpenStack and its releases have yet to be refined.

Cinder design sessions

I attended several design sessions for Cinder. And from what I saw, here is what the development community should focus on for the next few months:

  • Make the Cinder volume service highly available. Currently only active/passive deployment is possible because:
    • Local (file-based) locks are used when operating on Cinder volumes. This should be replaced by a distributed lock service (such as OpenStack Tooz).
    • Usage of “SELECT […] FOR UPDATE” SQL statements which don’t behave nicely (and deadlock) with MySQL Galera (the multi master SQL clustering solution).  This should be replaced by a “compare and swap” approach which guaranties atomicity, without deadlock, to prevent complications that may occur when multiple writers are working within the same row of a database table.
    • The internal Cinder state machine doesn’t handle very well the ‘*-ing’ states (when volume is in creating/deleting state)
  • Make the Cinder backup service more scalable and performant. With the current architecture, cinder-backup and cinder-volume services must be collocated on the same server. This limits the number of cinder-backup services that a cloud admin can deploy. Moreover, some performance problems are observed, backing up a volume takes a long time, instance. It could be related to the fact that data is compressed and optionally encrypted before being sent to the storage backend. Or it could be that a lot of data has to be moved around and the code/Python is not doing it efficiently. No one in the audience was really sure…
  • Make the Cinder drivers (such as ours based on Scality SOFS) programmatically publish, through an API endpoint, a set of well-defined capabilities they support. First, developers will need to come up with an agreement on what the terms “thin provisioning,” “QoS,” and “replication” really mean. Some terms are more controversial, so vendor drivers will also publish the “extra feature(s)” they support but in a separate namespace. The idea behind standard capabilities is to let cloud operators know what the storage backend supports without having to browse a documentation that can quickly become obsolete. The end users will also know the exact characteristics of the volume they are using/paying for.

The Hummingbird project – An alternative Swift object-server written in GoLang

Rackspace Cloud Files, based on OpenStack Swift, hosts dozens of petabytes (PBs) of data and tens of billions objects. Because of the huge size of the data, performance is critical, and that’s why a team at Rackspace decided to write a minimal Object Server in the Go language (which is compiled and statically typed as opposed to Python which is interpreted and dynamically typed). Some initial benchmarks have been made by Rackspace and the performance seems impressive! But they have to be confirmed independently and on a wider range of use cases. Currently, the object-server in Go has not reached feature parity with the original object-server, as it is currently lacking the support of middleware (extensions) as well as storage policies.  Additionally, some concerns about the lack of documentation, code comments, and a clear programming interface have been made.

Initial Hummingbird benchmark

Initial Hummingbird benchmark

What’s really interesting about the Hummingbird project is that, though in its infancy, the development is done in the open so everybody gets a chance to participate. It also raises the question of what language(s) are officially supported in the community; so far it’s been only Python.

The OpenStack Summit Talk Selection Process is Broken…

… or at least that’s what Mark Baker, one of the chairs for the “Enterprise” conference tracks for this summit, thinks. For the Vancouver summit, there were more than 1,000 talk submissions, which means future attendees of the summit could (and were expected to) vote for more than 1,000 talk abstracts!  That’s way too many to consider and also discouraging for those wishing to participate. In the end, the voting system favors big companies who send promotions for their talks through internal mailing lists, social media (through sites like Twitter), blog posts, etc. Moreover, the session feedback from previous summits is not taken into account to improve the next summit talk selections. There’s also pressure on vendors to get a lot of talks accepted so that they show how active in the community they are, what they do, how they think, what they contribute to, etc.

talk selection

The talk selection process

During his talk, Mark Bayer suggested to limit the number of proposals per person and/or company. But there was some kind of agreement in the audience that people will try to “game play” around the limit and it would prevent newcomers in the community from speaking because big companies would be tempted to only present their main/key speakers. Another suggestion was to strip out personal information about the speaker from the talk proposal, such as their name and company, so that votes will be based only on the content of the abstract. This idea seemed to make consensus in the audience, and hopefully the numerous people working for the OpenStack Foundation that were present at the time heard it too…

Closing words

As usual it was a great summit. The organization was flawless and the Vancouver Convention Center is awesome. On the more technical side, I felt nothing ground-breaking happened in the past six months (i.e. some design sessions from the last summit were slightly updated and served as is for this summit) and topics that were hot (high availability [HA], upgrade, containers) are still hot. But I guess that’s the price of “stabilization” and maturity.

Leave a Reply

Your email address will not be published. Required fields are marked *