armenzg: Hera Try Server Unit Tests status

The Mozilla@Seneca Hera Try Server has been able to run Unit Tests after uploading a patch during this week!

This is basically what it does:

• Upload a patch through thos form sendchange_test.cgi (You need credentials)
• The processchanges_test.pl in the build master checks the folder on the try server where the patch are and see if there is any one new. Right not it checks every 1 or 2mins but we will change this to incron (Upon new file creation - This means immediate response)
• The perl script copies all new patches locally and sends the buildbot a message "builbot sendchange"
• This triggers the buildbot to make the "tests" branch to get started
• It applies the patch to the latest source code, it builds, it uploads the build back to the try server through SCP and SSH (this allows the developer to download it) and it runs all 6 unit test suites against that build

Left to be done:

• Add incron for the processchanges_test.pl to run
• Put the waterfall view behind some security measures (We don't want random people to just "force builds" and loose CPU while we want identified users to be able to)
• Add ssh key pairs for the mac and the windows machine
• Create the slave on the windows machine
• Once done, clone all linux machines and the windows machines
• Modify the sendchange_test.cgi file to identify the user who uploaded the patch (right now, every patch says to be uploaded by "armenzg")
• Unify the sendchange_test.cgi form to allow users to upload patches and just get builds without "tests"
• It will be nice to notify the user when the builds are done OR when the unit tests are run (I do not know how)

Another idea - Regionalized builds

I believe that it would be a good project to allow the Hera server to spit back a regionalised build, using Rueen auto-localization tool. This obviously requires more research, but I believe it is viable project to try.

This post could have been much more technical and longer but I decided to KISS since I won't have time to spend time documenting until Tuesday or Wednesday.

armenzg: Where do I display the unit tests?

Unfortunately for me, the major system that I am implementing this semester has drained too many hours (~150hours accumulated) and that for being a SINGLE course is way too much. You can tell of this because the quantity of work I have generate for each release of my Mozilla project (integrate unit tests to Seneca's try server) is a one day worth of research and work. I expect that after this coming Wednesday (final run) I could rest of that project and dedicate myself to my Mozilla project and my upcoming exams.

For this release I will mention what I have tried and found out. Two major things that I have been working on and have to get done/fixed:

• The unit tests need a DISPLAY in which to show up. I tried to see what is going in the slaves and I have to do one of two things "get someone with root access to either set it to boot to runlevel 5, or do init 5 after it boots and that'll start up the desktop and you'll be able to log in via vmware console" OR "look up an Xvnc tutorial and figure it out (does not require root access :) )" (thanks a lot to rhelmer and I wish you the best!)
• Through "Seneca Buildbot Try Server - Unit Tests enabled (soon)"(for security reasons it needs credentials - Look at the screenshot) you will be able to upload a patch and will run the unit tests but for now I will have to see if the perl script is properly set up. So far I know that I can submit patches to the folder "patches" in the try server. This will have to be retrieved by the buildslave to apply it to the latest source code.
  

NOTE: If your force a build of the unit test column, you will see that I have removed a lot of the steps, just to be able to run the test as soon as possible

armenzg: Sess16 - Automated testing on Hera

This blog post announces that I have got my hands dirty adding the steps to do automated builds on buildbot with testings on the hera server.
I have got access this last past Tuesday and the last 4 weeks I was working on the buildbot configuration for CAIRO, and this is the summary of what I have tried to do:

• Access the buildmaster in hera.senecac.on.ca
• Modify the master.cfg to add the build steps for the linux machine
• Add mozbuild.py file and the mozconfig-linux-tests

My goal was to force a complete build with tests without ruining Adam's configuration (he set up the try server)

NOTE: This buildbot setup will allow to force builds with tests and builds without tests









NOTE: You can see the second yellow column that indicates that is building - check it yourself OR force a build (and go for a coffe)

Problems

As always, I keep a section for problems and solutions for future reference:

• I used "buildbot restart ." instead of "buildbot reconfig ." and therefore I did put the try server down :( - BTW, do not use "restart" if you want to just re-read master.cfg so your changes are applied, because instead of using the last working copy of master.cfg (as it says) it just dies and never starts again
• I had to put some of the environment variables (MozillaEnvironments['centos']) inside buildcustom/env.py file since it was defined in there and also in my code but did not catch it this way
  
• At some point my "mozconfig-linux" file contained CRLF and I would get "line 17 errors" when I had only 9 lines; I used file mozconfig-linux-tests to confirm and dos2unix mozconfig-linux-tests to fix it (thanks rhelmer) - I remember the "line" errors but I can't find the log that said it - When I do cat .mozconfig, you can actually see that there are extra new lines in the output)
• There are has been a lot of small problems that I have run into but not relevant

Conclusion

I am in the Hera server, I can make changes and I have got a build with automated testing.

What is left?

• Make the other machines to build too (I do not know if they are up and running)
  
• Learn how to make it build following the "tryserver" way (give a patch and run the build with tests)
• Find out why the compiling takes forever