Rally review dashboard

Reviewing process with Gerrit is very nice until you got more than 100 open patches on review, after that it becomes really hard task.

Fortunately  Gerrit allows you to build custom dashboards that allows to group changes by various criteria like patches that passed CI and don’t have -1 in code review. Making a properly URL that will build custom dashboard is quite hard task, but it can be simplified a lot if you use gerrit-dash-creator. Roman Vasylets did a great job and create a Rally Dashboard url:


Now all patches are grouped! Read more to see how patches are grouped…


Read More

Rally can generate load with passed users now!

Finally, I happy to announce that OpenStack Rally team, after more than 1 year of work, finished support of benchmarking with already existing users in OpenStack. This is crucial feature that simplifies adoption of Rally in enterprise world.

Why it’s so important?

There are 2 very important use cases from production world:

  1. It’s simpler to run Rally against production cloud
    Rally can use existing users instead of creating own which is impossible in case of r/o Keystone backends like LDAP and AD.
  2. It’s safer to run Rally against production cloud
    Rally can be run from isolated group of users and if something went wrong it won’t affect rest of the cloud users.


Read More

Rally v0.0.3 – What’s new?

Rally v0.0.3

New Features & API changes

  • Add the ability to specify versions for clients in benchmark scenarios
    You can call self.clients(“glance”, “2”) and get client initialized for specific API version.
  •  Add API for tempest uninstall
    $ rally-manage tempest uninstall    # removes fully tempest for active deployment
  • Add a –uuids-only option to rally task list
    $ rally task list –uuids-only    # returns list with only task uuids
  • Adds endpoint to –fromenv deployment creation
    $ rally deployment create –fromenv
    # recognizes standard OS_ENDPOINT environment variable
  • Configure SSL per deployment
    Now SSL information is deployment specific not Rally specific and rally.conf option is deprecated. Take a look at sample.


Read More

Rally “verify” as the control plane for Gabbi, Tempest & in-tree functional tests

It goes without saying that making OpenStack easy testable is crucial for future of OpenStack adoption. Let’s see how OpenStack testing process can be improved by Rally functional testing control plane.

OpenStack architecture pros & cons in nutshell

OpenStack has micro-services architecture. There are bunch of projects, each project has bunch of services, and these services are collaborating together to provide IaaS and PaaS. Micro services approach is not the silver bullet architecture that resolves all issues.

Benefits of Micro Services approach

  1. Isolation. Every part of system: VM, Volumes, Images, Object Storage is separated project, with separated API. So even if implementation is bad it can be rewritten without affecting other parts of system.
  2. Scale. Projects are developed separately, this means separated teams (with their experts and leads) are working on separated projects.

Issues of micro services approach

  1. Common functionality. If you would like to add new API method to all services, or to new
  2. Deployment configuration and management. You need to use separated projects that will just install and manage it. Like Fuel, RDO, JuJu.
  3. CI/CD. Testing requires very smart CI/CD that can pick proper versions of every project, configure all projects properly, start all services and then run tests.
  4. Testing. Every project requires test that brings big issues. Those issues and how to mitigate them is the goal of this blogpost.

Why it is hard to test OpenStack?


Read More

The simplest way to use OpenStack python clients

OpenStack is great but as all young projects it has some UX issues. One of the most user facing is initialization of OpenStack python clients. Every OpenStack Service like Nova, Cinder, Glance have own.

Since the begging in Rally we have internal class that unifies initialization of all clients. Recently we merged patch that allows to init this class from environment variables, which makes it really simple to use OpenStack python clients. Take a look at code snippet below:

boris@ubuntu:~$ . devstack/openrc admin admin

boris@ubuntu:~$ python
>>> from rally import osclients
>>> clients = osclients.Clients.create_from_env()
>>> clients.nova().flavors.list()
[<Flavor: m1.tiny>, <Flavor: m1.small>, <Flavor: m1.medium>, <Flavor: m1.large>, <Flavor: m1.nano>, <Flavor: m1.heat>, <Flavor: m1.xlarge>, <Flavor: m1.micro>]


Read More

Rally Tricks: “Stop load before your OpenStack goes wrong”

Benchmarking pre-production and production OpenStack clouds is not a trivial task. From the one side it’s important to reach the OpenStack cloud’s limits, from the other side the cloud shouldn’t be damaged. Rally aims to make this task as simple as possible. Since the very beginning Rally was able to generate enough load for any OpenStack cloud. Generating to big load was the major issue for production clouds, because Rally didn’t know how to stop the load until it was to late. Finally I am happy to say that we solved this issue.

With the new feature “stop on SLA failure” things are much better. 


Read More

Thoughts on making OpenStack community more user friendly

It’s well know that OpenStack is really fast growing community. More and more contributors are working on various features in more and more projects. Such interest produce a lot of scale issues for every OpenStack project team. Which sometimes produce fast and not enough thought up decisions. Some of those rules are making upstream work quite hard and annoying.

There are few places in OpenStack that could be rethink:

1. Projects should have feature request mechanism

So there should be easy way to propose new ideas to project community. At this point we have only two:

  1. Bugs on Launchpad
  2. Project-Specs repo

Both are not nice. Launchpad bugs are not reviewed/approved by PTL and core team. Project-specs are too complicated to write (you need to specify too much information related to implementation and how it affects project and who will work on and so on just to get it approved).

In Rally we tried a bit different approach called Feature request. So You actually shouldn’t write long poems. You should put just short description of what you need without deep technical overview.  Feature request shoot 2 birds with one stone: They are structured, unified and well reviewed (like specs), but as well they are very easy to propose (like bugs on launchpad)

2. Specs shouldn’t be required for everything

Specs are really nice approach for distributed teams to discuss huge changes/features in projects. But I really don’t think that they are required for any changes in project. Developers dislike bureaucracy and adding such steps makes their life harder and unhappier.

3. Project team should work on abandoned patches

Sometimes developers that were working on some features switch to other tasks and stop working on some patches. As a result patches are marked as abandoned and hidden from everybody eyes. I think it’s a huge mistake because such patches usually covers somebodies use cases. Finishing such patches mean being more oriented on end users requests.

4. Avoid -2 (“do not merge”) mark

There are few reasons why you shouldn’t use this mark:

  1. It demotivates author that spend a lot of time working on patch
  2. It is not user friendly. Instead of “help” contributor understands that he is not welcome.
  3. It’s not enough “open”. Nobody reviews patches with -2 mark. It means that only 1 person decide does this patch belong to project code or not. People are not robots they make mistakes, which means that good changes can be blocked forever.


Instead of making new layers of bureaucracy and new rules “how to avoid helping contributors”. Let’s try to figure out how to make community more friendly and improve contribution process. Reducing amount of specs, adding feature request and working on abandoned patches may be a good first step.

Read More