l10n repackaging - part 2

After 5 weeks in my internship and 2 weeks after my meeting with Axel Hecht. I have started to get some results that help us understand the problem that I am facing.

This is the bug that I am working on: "Bug 434878 - change how automation does l10n repacks" and here is the tar ball (you must have buildbot and others installed - here is a link with a script to help you out) that has a script that starts everything for you so you can see it running.

NOTE: I am reducing each section to focus in one part of the major problem, which will be mentioned in another blog posts

Main goal

• Change how l10n (localization) repackages happen

Reasons

• Not scalable. Currently we have to do ALL repackages even if we just want one
• Code difficult to follow
• Not easy to do big changes on how things get done

Ideas

• Use buildbot to help us
• Parallel the repackages. This will help especially to improve the process in Windows machines since it takes forever and a little more to finish the whole chunk of repackages
  

Obstacles

"Hello, I am a slave. What do you want me to do?"
Buildbot has a master who tells slaves what to do, but the issue is that the master gets configured at startup. Therefore for now:

• I have the list of locales hard-coded in the master's configuration file (this is not 100% true, but let's pretend it is)

Having the list of locales hard-coded means that I have to modify the configuration file and reconfigure the master again, which is not a good idea since there can be other builds happening AND more important than that is that it won't be automated but it will dependent on a human to logging to the master and reconfigure it.

How can we change this?

• An initial slave could do some "thinking" and notify "someone" (an object) in the master which locales to be repackaged
• An initial checks out a file, uploads it to the master and somehow notify the master to reconfigure itself

We will try to discard some options, find new ones and even try the "hacky" ones.
Using a slave, as Axel comments in the bug, is not appropriate for a slave to be saying what has to be built and what not.

This is the point where Axel and others have tried to bring a solution but the technology (buildbot) reaches its boundaries. Unless we have completely forgotten about a feature, we will have to come with a way around