Wednesday, November 25, 2009

Fennec's en-US single-locale binaries being uploaded to a different location

Up until now, Fennec's en-US deb file (566K) was being overwritten every time by the multi-locale's deb file (1.2MB) due to that the name of the files cannot be different. To avoid this we upload now the single-locale under an "en-US" sub-directory. under "latest" directories on ftp.

For more details:
* Bug 529140 - upload Fennec single-locale tar and deb files under "en-US" sub-directory




Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Friday, November 20, 2009

hy-AM (Armenian) moving forward

Robert Sargsyan has been localizing Firefox into Armenian for a really long time through Narro.
He recently has contacted me to get things rolling since he has translated 98-99% (94% according to compare-locales) of the strings.

It is now my turn to get into the technical details and move it to mercurial. These are the steps that we have taken:
  • Robert ported the strings from 3.5 to 3.6 (Narro allows you to do this)
  • Through Narro's interface I exported the project and downloaded the zip file that it generates
  • I checked out my clone of the Armenian 1.9.2 tree
  • I overwrote my tree with the contents of the zip file
  • I run compare-locales like this:
    compare-locales /Users/armenzg/moz/repos/mozilla-1.9.2/browser/locales/l10n.ini .. hy-AM-1.9.2
  • I removed the files that were indicated to be removed
  • I pushed my changes to my repository
What comes next (if I am not mistaken)?
  • generate a langpack
  • submit it to AMO (submit page)
  • promote the add-on
  • get people's review
  • convince drivers to give us commit access
  • push the changes to the official Mozilla hy-AM repositories
We won't make it for 3.6 and I can't wait to see what the future holds for this language.

Big thanks to Serge!



Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

libconic package needed for Maemo builds has been deployed

Thanks to puppet we were once again able to fix this easily.

All that puppet told the slaves to do is to run this command:
 su - cltbld -c '/scratchbox/moz_scratchbox -p apt-get --yes --force-yes install libconic0-dev'
 

What I did to fix this was:
  1. Check that a staging slave does not have that package "pkg-config conic --libs"
  2. Check that the file "targets/CHINOOK-ARMEL-2007/usr/include/conic/conic.h" does not exist
  3. Install it using a similar command that was used in a previous bug
  4. Check that "pkg-config conic --libs" returns what is expected "-lconic -ldbus-1 -lgobject-2.0 -lglib-2.0"
  5. Check that the conic.h exists where expected
  6. At this point we have a clear command to run by puppet and a "creates" argument.
  7. Deploy the patch in the staging-puppet server
  8. Login to another staging slave as root and run "puppetd --test --server staging-puppet.build.mozilla.org"
  9. Do checks 4 and 5 to see that the puppet changes took place
  10. Ask for review
  11. Commit and deploy to production-puppet
  12. Check a production like in step 8 and 9
  13. Check an hour later if a random slave has the change as well
Thanks to mfinkle for having written such a good description of the bug.

If you want to read more details check out "Bug 529462 - Add libconic package to Maemo build slaves"



Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Wednesday, November 18, 2009

Check for add-ons compatibility changes

I had my check for add-ons compatibility disabled but it recently stopped working.

To fix this instead of using this:
 extensions.checkCompatibility;false
you can use this:
 extensions.checkCompatibility.3.6b;false

As always, do not use this for stable releases as a workaround because one of your add-ons has not yet been updated after a new release. Doing so might make your new Firefox not to work as expected.




Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Thursday, November 05, 2009

My first push ever!

I received the ultimate power (just the build repos) on Monday and I got to use ot for the first time on Wednesday!
I was nervous I can't deny it but that didn't keep me from checking and rechecking (hg diff, hg st, hg in, hg pull -u, hg out) everything before pushing.

Note: I can only push through ssh and I had to setup a config file under .ssh.

armenzg:~ armenzg$ cat ~/.ssh/config
Host hg.mozilla.org
User armenzg@mozilla.com
armenzg:~ armenzg$ cat moz/repos/buildbotcustom/.hg/hgrc
[paths]
default = ssh://hg.mozilla.org/build/buildbotcustom

Here is my first changeset:
http://hg.mozilla.org/build/buildbotcustom/rev/015d0fabaa4e

Here is a couple of blog post from my coworkers that I found good to have:

Cool!



Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Wednesday, November 04, 2009

Firefox - Release Engineering infrastructure for Localization (L10n) - Blogging Blitz

In addition to our RelEng's blogging blitz I am going to blog about how our L10n release infrastructure works. In this first blog post I will explain our Firefox's coverage.

Scenarios

We currently have 3 scenarios:
  • Repackages on change
  • Nightly localized builds
  • Release localized builds

Deliverables

What our different deliverables are:
  • Langpacks (you can add a language as an add-on)
  • Installers (we generate a different one for each platform)
  • Complete and partial mars (these are generated for updating from one version to another one)
How many locales do we process:
  • More than 70 and depends on each branch
  • The number of locales per branch is determined by the all-locale's file of that branch

Key concepts

  • compare-locales = it compares en-US strings with the strings of a locale showing the differences between the two of them.
  • l10n-merged or merged = when a repackage is merged it means that if the locale has any missing strings then the English strings will be inserted. When we run compare-locales we tell it to create a "merged" directory in case the locale has any missing strings. When we create the installer if the "merged" directory exists the localized repackaged repackage will be based off that directory instead of the locales' repository. This ensures that it won't have any missing entities; avoiding this way any crash due to missing entities

How it works

Before I explain how each scenario works let me explain what makes them distinct:
  • compare-locale. We run compare-locales in all 3 scenarios.
  • l10n-merge. For releases we don't l10n-merge. We just use plain compare-locales.
  • triggering. Repackages-on-change are triggered by localizers' pushes while the nightly and releases are triggered when the en-US build finishes.
  • upload. Each scenario uploads to a different location (tinderbox-builds, nightly or releases)
  • updates. Only nightly and release scenarios have updates
How the repack-on-change scenario works:
  • We have a poller that polls for changes in all locale's pushlog and we poll every 15 minutes. The reason that we poll every 15 minutes is because it adds a huge load on the master if we did it in smaller periods of time. If we think about mc, 191 and 192 we poll more than 210 different pushlogs and that takes quite some load of our masters.
  • When a change for that locale is detected it triggers the triggerable L10n builds for all 3 platforms.
  • The deliverables for the locale are pushed to the tinderbox-builds directory on ftp.
How the nightly localized scenario works:
  • Instead of being triggered by a commit change on one of the locale's repositories, it is triggered by the en-US nightly build for that platform.
  • The nightly build triggers as many localized build jobs as the all-locale's file for that branch determines. This is accomplished in special buildbot schedulers that make use of the L10nMixin class by submitting a job per locale.
  • The repackages are l10n-merged!
  • The installers are uploaded to the nightly directories on ftp.
  • Unlike the repack-on-change scenario we have updates for this type of builds and that is why you will get to see complete mar files under the latest--l10n dir on ftp and partial mar files under the dated directories on ftp (NOTE that we don't keep the dated directories of localized builds forever as with the en-US builds).
 How the release localized scenario works:
  • This scenario is pretty similar to the nightly one but it has few differences.
  • The locales involved in the release are determined by shipped-locales and l10n-changesets and the revisions for each locale are specified in there.
  • The repackages are not l10n-merged.
  • We run l10n_verification to cross-check and make sure the repacks worked properly for the release builds.
  • We upload each locale's installers in its own subdirectory rather than all of them under an "-l10n" directory.  For instance you can see all the Windows localized builds for 3.5.4 in here. You can also note that en-US is in its own directory and all the langpacks under the xpi directory.
I know I have gone way into details but I believe it is worth having it recorded all together for once and for all. More to come on my next blog with regards to Fennec!

EXTRA: What builders we do have:
  • Firefox mozilla-{central, 1.9.0, 1.9.1, 1.9.2} {linux, macosx, win32} l10n build <- repack-on-change runs
  • Firefox mozilla-{central, 1.9.0, 1.9.1, 1.9.2} {linux, macosx, win32} l10n <- nightly runs
  • {linux, macosx, win32}_repack <- release runs
  • l10n_verification <- cross checking



Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.