Sunday, May 18, 2008

l10n repackaging - part 1

My name is Armen Zambrano Gasparnian and I am one of the two release engineering interns with Lukas Blakk. My current project is taking me to research on how the l10n repackages get done (this process generates the l10n builds for all different languages) and make improvements to it

I will describe the situation and some of the problems that we have currently:
  • l10n nightlies happen in old dedicated machines instead of running in generic buildslaves. This means if the machine fails, there are no nightlies at all. (Read more about inconveniences of not using generic machines in John O'Duinn's blog)
  • l10n nightlies are not been kept in our servers (unless I am mistaken); for each locale the only one build available is the "latest" one (latest-trunk-l10n, latest-Mozilla1.8-l10n) which gets overwritten every night. By being "kept" means that we don't keep the l10n build of a specific day, even though in the tinderbox's log you always get a URL, to grab the file (log, e.g. nightly/2008-05-17-09-trunk-l10n)
  • The previous might be a desired effect, since l10n repackages are not computational costly but storing them it is, all you have to do to get an l10n build is ... ... ... ... ... (I am sorry, it is not that straight forward). I have written this past Friday a little script that does some parts of that process for an specific locale, in this case, "af" (--->create locale script<---). The mozconfig used from the source code has an l10n tag (There are 4 tags: l10n, l10n_release, MOZILLA_1_8_l10n, MOZILLA_1_8_BRANCH_l10n_release). From running the previous script, I obtain the "af" .dmg file (inside of browser/dist) and the af.xpi (browser/dist/install/mac-xpi), which are the files that I want for a mac
One of our first goals is:
  • We want to make l10n repacks to happen in such way that can be scalable. Currently it works like this (which is not scalable):
  1. We do a "make configure"
  2. Download all locales
  3. Download from nightly/latest-trunk the latest en-US archive file (Isn't it dangerous to download something that can be overwritten while downloading it??)
  4. Do the repack for all locales (~1min per locale)
  5. After that, we upload all locales

Solutions:
  • A better way would be to break this down by 1) creating a locale 2) upload it and 3) announce it. This would allow to do l10n repackaging in parallel which would be a great improvement
  • This would also allows to generate "just" one locale if needed. Right now, we have to generate all of them even if we just want one

I don't want to prolong this by explaining how the l10n repackaging works currently but I will do so when we file this week the bug that will cover this project and will allow more discussion

NOTE = Just FYI we do l10n repackages when we have releases in the automation system instead of using the old tinder client machines and is just slightly different by using Bootstrap/Step/Repack.pm