Tuesday, September 16, 2014

Which builders get added to buildbot?

To add/remove jobs on tbpl.mozilla.org, we have to modify buildbot-configs.

Making changes can be learnt by looking at previous patches, however, there's a bit of an art to it to get it right.

I just landed a script that sets up buildbot for you inside of a virtualenv and you can pass a buildbot-config patch and determine which builders get added/removed.

You can run this by checking out braindump and running something like this:
buildbot-related/list_builder_differences.sh -j path_to_patch.diff

NOTE: This script does not check that the job has all the right parameters once live (e.g. you forgot to specify the mozharness config for it).

Happy hacking!


Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Thursday, September 11, 2014

Run tbpl jobs locally with Http authentication (developer_config.py) - take 2

Back in July, we deployed the first version of Http authentication for mozharness, however, under some circumstances, the initial version could fail and affect production jobs.

This time around we have:

  • Remove the need for _dev.py config files
    • Each production config had an associated _dev.py config file
  • Prevented it from running in production environment
    • The only way to enable the developer mode is by appending --cfg developer_config.py
If you read How to run Mozharness as a developer you should see the new changes.

As quick reminder, it only takes 3 steps:

  1. Find the command from the log. Copy/paste it.
  2. Append --cfg developer_config.py
  3. Append --installer-url/--test-url with the right values
To see a real example visit this


Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Thursday, August 07, 2014

mozdevice now mozlogs thanks to mozprocess!



jgraham: armenzg_brb: This new logging in mozdevice is awesome!
armenzg: jgraham, really? why you say that?
jgraham
: armenzg: I can see what's going on!
We recently changed the way that mozdevice works. mozdevice is a python package used to talk to Firefox OS or Android devices either through ADB or SUT.

Several developers of the Auto Tools team have been working on the Firefox OS certification suite for partners to determine if they meet the expectations of the certification process for Firefox OS.
When partners have any issues with the cert suite, they can send us a zip file with their results so we can help them out. However, until recently, the output of mozdevice would go to standard out rather than being logged into the zip file they would send us.

In order to use the log manger inside the certification suite, I needed every adb command and its output to be logged rather than printed to stdout. Fortunately, mozprocess has a parameter that I can specify which allows me to manipulate any output generate by the process. To benefit from mozprocess and its logging, I needed to switch every subprocess.Popen() call to be replaced by ProcessHandler().

If you want to "see what's going on" in mozdevice, all you have to do is to request the debug level of logging like this:
DeviceManagerADB(logLevel=mozlog.DEBUG)
As part of this change, we also switched to use structured logging instead of the basic mozlog logging.

Switching to mozprocess also helped us discover an issue in mozprocess specific to Windows.

You can see the patch in here.
You can see other changes inside of the mozdevice 0.38 release in here.

At least with mozdevice you can know what is going on!


Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.