armenzg: Session 15: When things go wrong

I was trying to write cairobuild.py as a file containing steps commands which derived from ShellCommand so I could have logically related steps grouped together. For instance, "autogen.sh", "make" and "make install", since all of them have to run to count as a "good" build. Another reason was to have a summary of the tests run, instead of having a look at the STDIO to see the results of the tests.

Well, I decided to make it happen at school were we have computers (being setup by us, here and there) for our Open Source Projects, but things went wrong:

1. I try to get buildbot and twisted on the mac mini
2. I want to use package manager: macPorts cannot reach its rsync server; fink can reach a cvs repository to selfupdate but takes for ever
3. I decided to get tarball and build it myself; Buildbot only works with Twisted 2.5 but I would only have 2.4 even tough I build it (sudo python ./setup.py install)
4. I realized that buildbot got installed under /Libraries/python/2.5/site-packages (I think) and twisted too, but it seemed to don't catch it as 2.5
5. I found through a blog post (I left the link at school) talking about it and it seemed I had to export PYTHONPATH to point at the right place
6. Now I should had been ready, but when I tried to start my buildbot master I got this message:
   

   Traceback (most recent call last):
   File "/Library/Python/2.5/site-packages/twisted/application/app.py", line 379, in run
   runApp(config)
   File "/Library/Python/2.5/site-packages/twisted/scripts/twistd.py", line 23, in runApp
   _SomeApplicationRunner(config).run()
   File "/Library/Python/2.5/site-packages/twisted/application/app.py", line 157, in run
   self.application = self.createOrGetApplication()
   File "/Library/Python/2.5/site-packages/twisted/application/app.py", line 207, in createOrGetApplication
   application = getApplication(self.config, passphrase)
   --- ---
   File "/Library/Python/2.5/site-packages/twisted/application/app.py", line 218, in getApplication
   application = service.loadApplication(filename, style, passphrase)
   File "/Library/Python/2.5/site-packages/twisted/application/service.py", line 341, in loadApplication
   application = sob.loadValueFromFile(filename, 'application', passphrase)
   File "/Library/Python/2.5/site-packages/twisted/persisted/sob.py", line 215, in loadValueFromFile
   exec fileObj in d, d
   File "buildbot.tac", line 9, in
   BuildMaster(basedir, configfile).setServiceParent(application)
   File "/Library/Python/2.5/site-packages/buildbot/master.py", line 370, in __init__
   self.status = Status(self.botmaster, self.basedir)
   File "/Library/Python/2.5/site-packages/buildbot/status/builder.py", line 1779, in __init__
   assert os.path.isdir(basedir)
   exceptions.AssertionError:

And this is where I basically left it and I should find why next time I go to school or just choose one of the Linux computers and run it there, but I can't promise that everything will be easy.

CairoTests class

class CairoTests(ShellCommand):
   name = "Run Cairo Tests"
   warnOnFailure = True
   description = ["running tests"]
   descriptionDone = ["tests completed"]
   command = ["make", "test"]
   workdir = "cairo"
 
   def createSummary(self, log):
       passCount = 0
       failCount = 0
       for line in log.readlines():
           if "PASS" in line:
               passCount = passCount + 1
           if "FAIL" in line:
               failCount = failCount + 1
       summary = "TestsResults:" + str(passCount) + "/" + str(failCount) + "\n"
       self.addCompleteLog('summary', summary)
  
   def evaluateCommand(self, cmd):
       superResult = ShellCommand.evaluateCommand(self, cmd)
       if SUCCESS != superResult:
           return WARNINGS
       if None != re.search('FAIL', cmd.logs['stdio'].getText()):
           return WARNINGS
       return SUCCESS

After this blog post I will have to change this code because I believe it does not capture properly how many tests failed or passed.
Cairo's tests have the keywords PASS, XFAIL, UNTESTED and FAIL at the end of some lines and the summary of each group of tests is after the block of tests and appears at the beginning of the line followed by a semicolon. Check the next quoted text:

TESTING svg-surface-source
Test using various surfaces as the source
svg-surface-source-image-argb32 [0]: PASS
svg-surface-source-image-argb32 [25]: PASS

<-- There are more lines but nothing relevant -->

svg-surface-source-xlib-fallback-rgb24 [0]: UNTESTED
Failed to open display: :0.0
Failed to open display: :0.0
svg-surface-source-xlib-fallback-rgb24 [25]: UNTESTED
PASS: svg-surface-source <-- This is what I call the summary of the block of tests

We also have at the end of the STDIO that shows the results of all tests and my summary should match that OR maybe I should just show that in the summary instead:

========================================================================
7 of 156 tests did not behave as expected (4 unexpected passes)
Please report to http://bugs.freedesktop.org/enter_bug.cgi?product=cairo
========================================================================
make[4]: *** [check-TESTS] Error 1
Failed tests:
    ft-text-vertical-layout-type1: image test-fallback test-meta test-paginated
Failures per surface - image: 1, test-fallback: 1, test-meta: 1, test-paginated: 1.

Next releases

It seems that the Hera Try Server will be up and running something really soon and I will have to make sure that the tests are being run properly and that it meets the desires of the Mozilla community. Adam has been doing a great job and as I have heard from Shaver they will be really happy to see this working.

In my next release I will expect to have the Cairo steps for Windows and the performance tests (which I thought I could have got done for this week but did not happen) and whatever is required for me to work on the Try Server