Running Gaia UI b2g panda tests on the Cedar branch

Back in November, Release Engineering tackled with IT, the A-team and the B2G team, the undertaking of running in production the Gaia UI tests on the b2g pandas [1].

Back in December 7th, we managed to get all the piping done for re-imaging pandas and assigning dummy jobs to them (see "B2G test jobs running on panda boards on the Cedar branch" post for details).

Last week, after I came back from holidays, I was pleased to see that one of the last blocking bugs had been solved (bug 820617 - Add a hook to make NetworkManager not manage offline status and use it in Marionette for B2G CI) and I could now try to run the tests.

Last Friday, we landed the code to actually run the tests [2]. This is simply run with a mozharness script like this:

/tools/buildbot/bin/python scripts/scripts/b2g_panda.py --cfg b2g/panda_releng.py

At that point in time we stopped running dummy jobs and started running tests for the first time.

Since then, then test jobs have been failing but we are working on the last few bugs to get them to be green. See bug bug 829053 (Gaia UI tests failing with JavascriptException: TypeError: settings is null) if you're curious.

Thirty five fixed dependent bugs later (and 2 bugs left to be fixed) I can say that it has been a great joint project and I have personally learned a lot through it.

It would have great to have completed it a while ago but too many difficulties and code problems were found.

I hope to write to you soon once the tests go green and we start running them every b2g branch.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=802317
[2] http://hg.mozilla.org/build/mozharness/rev/caf3b92e7207

---

This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

B2G test jobs running on panda boards on the Cedar branch

(This was posted on Dec. 7th but it seems I forgot - thanks edmorley for noticing - to add the "planet" tag so it would show up on planet.mozilla.org)

Hi,
I want to announce that we recently started running test jobs on the Panda ES boards. These are running on tbpl on the Cedar branch:
https://tbpl.mozilla.org/?tree=Cedar&jobname=b2g_panda&noignore=1

These jobs are currently not running real tests but testing the plumbing. It tests that we can image the boards and that they come back up.

This is how the process work:

• We detect that a new b2g panda image has been uploaded to a private location
• We give the job to a buildbot instance on one of our servers
• The buildbot instance, through API calls, requests the specific device associated to it from Mozpool [2]
• Mozpool downloads the b2g artifacts and re-images the device
• Buildbot waits for a fixed amount of time for the device to be "ready"
• In later models it will be able to request "any" device rather than a specific model
• Once the device is available we run a small test which is to cat /system/b2g/application.ini 
• This shows from which build it got re-imaged and that we can run something on the board
• Once we run that test we return the device to Mozpool and finish the job

Here's the log of our first working job in production:
https://tbpl.mozilla.org/php/getParsedLog.php?id=17716049&tree=Cedar&full=1

Our first candidate for tests running on the pandas are the Gaia UI tests [3]. Right now, those tests can not be run remotely but the work of getting it working is getting closer to be completed.

While that happens, on the releng side we will be fixing issues we're finding on the Buildbot <-> Mozpool <-> Panda interactions. Not all green jobs are yet real successful jobs. We should recover better from the failing imaging jobs.

This is a very neat and improved imaging system by introducing Mozpool to our systems (Which is a mix of BlackMobileMagic - aka bmm -, LifeGuard and Mozpool). The A-team and IT developed this great system.

Due credit to aki, ahill, Callek, catlee, ctalbert, digi, dustin, dividehex, nthomas, jgriffin and jhopkins for getting us this far.

More to come; stay tuned!

[1] http://pandaboard.org/node/300/#PandaES
[2] https://wiki.mozilla.org/ReleaseEngineering/Mozpool
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=802317

---

This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Juvenile diabetes? You can help!

Hi,
Every day many of our friends and relatives face diabetes.
My friend Greg has been cycling to Niagara Falls for the last 6 years raising awareness and donations for diabetes research.
Please consider donating to this cause. Every little counts:
http://jdrfca.donordrive.com/index.cfm?fuseaction=donorDrive.participant&participantID=164692

---

This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.