armenzg: Sess9 - Test integration build steps

Building with Buildbot requires to set up a file master.cfg with instructions to be sent to each buildslave to build or do whatever you want.

These steps are constructed by using the method addStep() of a BuildFactory and it goes like this:

myFactory = facory.BuildFactory()
myFactory.addStep( ........) //You do it multiple times until you have all steps and then...

// You set up a Builder for that build slave
firefox_trunk_centos5_builder = {
'name': "Linux qm-centos5-01 dep unit test",
'slavenames': ['linux'],
'builddir': "trunk_centos5",
'factory': myFactory
'category': "Firefox"}

What I am going to show you now is the steps required to do a build with running different test suites, but instead of the code I will just mention the classes passed to the addStep() method and where that step is executed (there are more attributes and parameters, if you want to see the full steps click on this link):

NOTE: If not specified the working directory is: "mozilla"

• MozillaCheckoutClientMk workdir="."
• FileDownload //To get the .mozconfig file
  
• ShellCommand -> "cat .mozconfig"
• Compile -> "make -f client.mk checkout"
• MozillaClobber //I am not sure why Clobber is for
• Compile -> "make -f client.mk build"

Up to here nothing special, at this point, after the build, we will have to run all of the tests:

• 1) MozillaCheck -> workdir="mozilla/objdir"
• 2) MozillaUnixReftest -> workdir="mozilla/layout/reftests"
• 3) MozillaUnixCrashtest -> "mozilla/testing/crashtest"
• ShellCommand -> "rm places.sqlite" workdir="/home/buildbot/.mozilla/firefox/vrttezm8.default" //I do not know why this step, maybe this master.cfg was specified to unit test with PLACES enabled
• 4) MozillaMochitest -> workdir="mozilla/objdir/_tests/testing/mochitest"
  
• 5) MozillaMochichrome -> workdir="mozilla/objdir/_tests/testing/mochitest"
  
• 6) MozillaBrowserChromeTest -> workdir="mozilla/objdir/_tests/testing/mochitest"
• And that's it!
  

You also pass to each test step the parameter env=MozillaEnvironment['centos'] that contains all environment variables for this specific builder and its specific build steps.

Note about the classes passed to addStep()
All of the classes passed to the addStep() method inherit from the class ShellCommand, I will show you:

• In mozbuild.py we have this line:
  from buildbot.process.step import ShellCommand
• Then when we define one of the classes:
  class MozillaReftest(ShellCommand): //I believe this indicates inheritance in Python
  class MozillaUnixReftest(MozillaReftest): //This is a subclass of MozillaReftest, which is used in one of our steps
• The test classes define these two methods, which are to evaluate the command and to create a summary:
  def createSummary(self, log): //This method calls addCompleteLog(), which I believe sends a summary to the status of the build
  def evaluateCommand(self, cmd):

Conclusion

After this blog post, I am closer to understand every step required to run all automated tests but I still have to understand how this will work together with a "try server" set up, since what I have is an automated build with dependencies and the "try server" works upon check in and I believe it has to be without dependencies.