l10n repackaging - part 4 - packager.mk and rewriting goals

Accomplished

• I have finished all that it is required to generate a locale's dmg file and its associated xpi (an add-on to change the language of your browser) under a buildbot configuration
  
• I have filed a bug: "Bug 438240 - packager.mk does not mount image automatically"
• This line used to be "good enough" to answer in an automated way to the question: "do you want to mount this image?"
  echo Y | hdiutil attach -readonly -mountroot /tmp -private -noautoopen $(UNPACKAGE) > hdi.output;
  but it is not good enough at least for Leopard and according to Axel for Tiger (We have decided to open a zoo instead of continuing with our l10n work in Mozilla ;)
• "My fix" (which is based in the scripts from Axel and Alice) is an expect script which feeds answers to an spawned process (hdiutil) which in reality would have expected a human to reply to the questions
• I have added a 2 line patch to fix this but I have no rush to get it in the tree
• I have tested something John wanted me to try and found that it did not work as he wanted.
• I started my buildbot configuration with 3 slaves at the same time
• The 1st slave picks "af" as the locale to work on
• The 2nd slave picks "ar" as the locale to work on
• The 3rd slave picks "be" as the locale to work on
• I killed the 2nd slave -- therefore "ar" was not completed
• When the 1st slave was done it picked up "bg" as the next locale to work on instead of the unfinished "ar"
  
• As you can see the locale "ar" would have never been processed in this run
• John says that he worked at some point with buildbot and saw that unfinished BUILDS where returned to the queue and taken by the next available slave. Since my Build and PeriodicL10n classes are "custom-made", they might be some features missing
  

Rewriting tasks/goals

Here is the list of problems/tasks/goals that I have, the type of problem that they are and the priority that they should be given:

• P1-DEFECT - Use buildbot sendchange to FORCE the processing of a single locale
• P2-DEFECT - Use same timestamp as en-US
• P3-DEFECT - If locale in process and proccess not completed it should be put back into the queue so it can be reassigned
  
• P3-CONFIG - Triger l10n repackages after succesful en-US build
• P3-PERF - How to deal with common steps? They could be executed only once before processing the first locale
• P3-PERF - When to clobber? There are various scenarios to consider
• P4-CONFIG - A lot of small steps should be unified under a module in buildbot custom
• P4-CONFIG - Add configuration to staging 1.8
• P4-CONFIG - WGET - what is the exact URL of the latest en-US to get?
• P4-CONFIG - compare-locales.py
• P4-CONFIG - l10n verify - I have seen that being run in production, more research to be done
• P4-CONFIG - Push && Announce - After each locale is processed, they have to be pushed to an FTP server and being announced via email or other

What does these priorities mean? I see 2 types of problems in here.

1. Problems that require research, time and a lot of thinking since they are not implemented anywhere else OR I do not know how to do it
   
2. Problems that require less brain power and it is just a matter of putting pieces together of concepts that are being used somewhere else

The first type of problems are the ones as priority 1 or 2, which have more of my attention and dedication. The other ones could be easily solved by anyone on Mozilla without too much effort.

There are more problems to be solved in l10n-build but this list just narrows it to what is required to move from the tinderbox infrastructure to buildbot.

Any feedback and questions are welcomed.