Thursday, November 06, 2014

Setting buildbot up a-la-releng (Create your own local masters and slaves)

buildbot is what Mozilla's Release Engineering uses to run the infrastructure behind
buildbot assigns jobs to machines (aka slaves) through hosts called buildbot masters.

All the different repositories and packages needed to setup buildbot are installed through Puppet and I'm not aware of a way of setting my local machine through Puppet (I doubt I would want to do that!).
I managed to set this up a while ago by hand [1][2] (it was even more complicated in the past!), however, these one-off attempts were not easy to keep up-to-date and isolated.

I recently landed few scripts that makes it trivial to set up as many buildbot environments as you want and all isolated from each other.

All the scripts have been landed under the "community" directory under the "braindump" repository:

The main two scripts:

If you call with -w /path/to/your/own/workdir you will have everything set up for you. From there on, all you would have to do is this:
  • cd /path/to/your/own/workdir
  • source venv/bin/activate
  • buildbot start masters/test_master (for example)
  • buildslave start slaves/test_slave
Each paired master and slave have been setup to talk to each other.

I hope this is helpful for people out there. It's been great for me when I contribute patches for buildbot (bug 791924).

As always in Mozilla, contributions are always welcome!

PS 1 = Only tested on Ubuntu. If you want it to port this to other platforms please let me know and I can give you a hand.

PS 2 = I know that there is a repository that has docker images called "tupperware", however, I had these set of scripts being worked on for a while. Perhaps someone wants to figure out how to set a similar process through the docker images.

Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.


  1. This is clearly inferior to my own way of setting up and running a buildbot instance:

    1. Set up a master by following various snippets found here and there.
    2. Get a slave loaner. Never ever give it back.
    3. SSH into the slave, setting up a massive hairball of port forwardings via .ssh/config
    4. Trick the slave into not rebooting constantly by symlinking /sbin/reboot to /bin/true.
    5. Discover that the slave has somehow overwritten its buildbot.tac again. Restore it from a copy I stashed away on some backup disk that I thought I didn't need any more. Edit it to lie about its name according to the latest naming scheme in the buildbot master config.
    6. Discover that a new entry is now necessary somewhere in the master config. Realize that I have completely and totally forgotten how to set up a master from scratch. Flail around randomly until I stumble across Hack in the new setting.
    7. Maintain a pile of local patches that do things like
    - pointing buildbot to pull from my mozharness, tools, etc.
    - turn off the hg poller so I don't keep getting jobs triggered from other people's pushes
    - massacre most of the other slaves so I can find mine in the slave list and the pages don't take too long to load
    - same for the builders

    If you could do it my way, why would you want your scripts? :-)

    1. I think I should put my post down and backout the code ;)