Friday, March 07, 2008

armenzg: Session 15: When things go wrong

I was trying to write as a file containing steps commands which derived from ShellCommand so I could have logically related steps grouped together. For instance, "", "make" and "make install", since all of them have to run to count as a "good" build. Another reason was to have a summary of the tests run, instead of having a look at the STDIO to see the results of the tests.

Well, I decided to make it happen at school were we have computers (being setup by us, here and there) for our Open Source Projects, but things went wrong:

  1. I try to get buildbot and twisted on the mac mini
  2. I want to use package manager: macPorts cannot reach its rsync server; fink can reach a cvs repository to selfupdate but takes for ever
  3. I decided to get tarball and build it myself; Buildbot only works with Twisted 2.5 but I would only have 2.4 even tough I build it (sudo python ./ install)
  4. I realized that buildbot got installed under /Libraries/python/2.5/site-packages (I think) and twisted too, but it seemed to don't catch it as 2.5
  5. I found through a blog post (I left the link at school) talking about it and it seemed I had to export PYTHONPATH to point at the right place
  6. Now I should had been ready, but when I tried to start my buildbot master I got this message:
    Traceback (most recent call last):
    File "/Library/Python/2.5/site-packages/twisted/application/", line 379, in run
    File "/Library/Python/2.5/site-packages/twisted/scripts/", line 23, in runApp
    File "/Library/Python/2.5/site-packages/twisted/application/", line 157, in run
    self.application = self.createOrGetApplication()
    File "/Library/Python/2.5/site-packages/twisted/application/", line 207, in createOrGetApplication
    application = getApplication(self.config, passphrase)
    --- ---
    File "/Library/Python/2.5/site-packages/twisted/application/", line 218, in getApplication
    application = service.loadApplication(filename, style, passphrase)
    File "/Library/Python/2.5/site-packages/twisted/application/", line 341, in loadApplication
    application = sob.loadValueFromFile(filename, 'application', passphrase)
    File "/Library/Python/2.5/site-packages/twisted/persisted/", line 215, in loadValueFromFile
    exec fileObj in d, d
    File "buildbot.tac", line 9, in
    BuildMaster(basedir, configfile).setServiceParent(application)
    File "/Library/Python/2.5/site-packages/buildbot/", line 370, in __init__
    self.status = Status(self.botmaster, self.basedir)
    File "/Library/Python/2.5/site-packages/buildbot/status/", line 1779, in __init__
    assert os.path.isdir(basedir)
And this is where I basically left it and I should find why next time I go to school or just choose one of the Linux computers and run it there, but I can't promise that everything will be easy.

CairoTests class

class CairoTests(ShellCommand):
name = "Run Cairo Tests"
warnOnFailure = True
description = ["running tests"]
descriptionDone = ["tests completed"]
command = ["make", "test"]
workdir = "cairo"

def createSummary(self, log):
passCount = 0
failCount = 0
for line in log.readlines():
if "PASS" in line:
passCount = passCount + 1
if "FAIL" in line:
failCount = failCount + 1
summary = "TestsResults:" + str(passCount) + "/" + str(failCount) + "\n"
self.addCompleteLog('summary', summary)

def evaluateCommand(self, cmd):
superResult = ShellCommand.evaluateCommand(self, cmd)
if SUCCESS != superResult:
if None !='FAIL', cmd.logs['stdio'].getText()):
return SUCCESS

After this blog post I will have to change this code because I believe it does not capture properly how many tests failed or passed.
Cairo's tests have the keywords PASS, XFAIL, UNTESTED and FAIL at the end of some lines and the summary of each group of tests is after the block of tests and appears at the beginning of the line followed by a semicolon. Check the next quoted text:

TESTING svg-surface-source
Test using various surfaces as the source
svg-surface-source-image-argb32 [0]: PASS
svg-surface-source-image-argb32 [25]: PASS

<-- There are more lines but nothing relevant -->

svg-surface-source-xlib-fallback-rgb24 [0]: UNTESTED
Failed to open display: :0.0
Failed to open display: :0.0
svg-surface-source-xlib-fallback-rgb24 [25]: UNTESTED
PASS: svg-surface-source <-- This is what I call the summary of the block of tests

We also have at the end of the STDIO that shows the results of all tests and my summary should match that OR maybe I should just show that in the summary instead:

7 of 156 tests did not behave as expected (4 unexpected passes)
Please report to
make[4]: *** [check-TESTS] Error 1
Failed tests:
ft-text-vertical-layout-type1: image test-fallback test-meta test-paginated
Failures per surface - image: 1, test-fallback: 1, test-meta: 1, test-paginated: 1.

Next releases

It seems that the Hera Try Server will be up and running something really soon and I will have to make sure that the tests are being run properly and that it meets the desires of the Mozilla community. Adam has been doing a great job and as I have heard from Shaver they will be really happy to see this working.

In my next release I will expect to have the Cairo steps for Windows and the performance tests (which I thought I could have got done for this week but did not happen) and whatever is required for me to work on the Try Server


  1. regarding the mac mini error - I too am on a mac mini, but the error was caused by creating the buildslave in wrong directory using the buildbot create-slave command and then moving it to where I wanted it using mv. There is a path in the buildbot.tac file that needs to be adjusted in that case. Nothing to do with the platform at all. HTH.

  2. It has been such a long time since my first baby steps... good old memories...

    Hey Jim thanks for the help! Yeah as you say moving the buildslave without rectifying buildbot.tac is an error by default