en-US nightly updates for win32 not as timely as it used to be

A side effect of enabling the l10n updates is that en-US nightly builds that finish after 7AM PDT (which is when l10n nightly repackages are triggered) will take longer to receive an update offer. Specifically, this happens mainly for win32 builds since they take way longer to finish.

Currently there is only one machine that checks for pending updates to be generated and it generates them all pending jobs in one shot, pushes the snippets live and then checks again for more pending jobs.

The problem is that if the system picks X number of pending updates it will process them one by one but without pushing any snippets and partial MAR files live until it finishes with all X jobs.

Therefore, since the en-US nightly for win32 finishes after 7AM, it falls in the bucket of pending jobs and it can take two or three hours before the snippets are pushed live.

To fix this regression we have decided to have a temporary fix to our update generation system until we do the long term infrastructure changes.

To follow the work on this fix please follow the work on bug 511901.

We are really sorry for any inconvenience and we would like to have it fixed as soon as possible.

---

This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Updates + Nightly != just en-US

We finally have updates for all the locales we support instead of just en-US. This was enabled past Tuesday (right now only for mozilla-central) but not announced until we had it completely working.

Having updates with l10n-merge allow us to update users from a working l10n nightly build to the next working nightly.
This will help localizers to create a community of nightly testers and they won't have to worry for
A) having to tell their testers to manually download the latest nightly of their locale and/or B) worry that if their testers would have a nightly with missing entities (which meant a crashing experience).

This is a really short post but it has been by no means a short ride. Chris Cooper, Axel and I have been working for the last fifteen months on this project since I started as an intern. We had to untangle the logic from old tinderbox code, port it to buildbot, make buildbot do things that was not designed to do, unveil the "dark magic" of our updates generation system and fight any build ghosts trying to take us down.

Dramatization aside big thanks to Chris and Axel for working hard with me (and having patience), John O'Duinn and Seth for supporting us, Nick Thomas for the headaches I gave him, Ben Hearsum for helping understanding our snippet generation system and anyone from the community who helped me learn what I needed to learn.Last thanks for Mike Morgan with the AUS/Mac debugging.

I hope this contributes the L10n community's success in their respective countries and towards making more people support an open web!

PS = Party tonight at my place to celebrate :)

---

This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.