How to show subdirectories in your hg local setup

In my previous blog post I had a *wish list* section which had become a *I am blocked in here* section

Before I was using [collections] which I believe it saves you the hassle of having to declare individually each repo living within it but it does not show "l10n-central" in the URL which I need for my local development setup to repack on changes.

This is the hgweb.config that I am using now:

[web]
templates = /repos/hg_templates
style = gitweb_mozilla
[paths]
l10n-central/af = l10n-central/af
l10n-central/de = l10n-central/de
mozilla-central = mozilla-central

and the URLs look like this now (in bold those that are real repos):

• http://localhost:8000/
• http://localhost:8000/mozilla-central/
• http://localhost:8000/l10n-central/ <- It shows you the list of repos within it: "af" and "de"
• http://localhost:8000/l10n-central/af
• http://localhost:8000/l10n-central/de

I believe it is not the correct/proper way of doing it but I believe it should meet my needs now.

I still have a couple of questions regarding to:

1. why does the gitweb_mozilla style get applied in the real repos and not in the main page (http://localhost:8000) the same way that in hg.m.o?
2. why does http://localhost:8000/l10n-central/af show beside the title "summary" show "l10n-central/af" instead of "af' like in http://hg.mozilla.org/l10n-central/af?

setup a couple of l10n-central's repos locally *with* pushlog

This is the milestone that I wanted to reach after the last couple of blog posts.
I wanted to have a local hg server with few of the l10n repositories and with pushlog to be able to test an HgPoller. Let's see if this is the last one of this series of posts.

What I did on the server side:

1. cd /var/hg/l10n-central
2. hg clone http://hg.mozilla.org/l10n-central/af
3. hg clone http://hg.mozilla.org/l10n-central/de
4. replaced the .hg/hgrc of both repos with what was explained on my first post plus adding the push_ssl and allow_push values:
   

   # the a_ and z_ prefixes are for ordering:
   # any hooks which can veto an action (require-singlehead) should be run before
   # hooks which make permanent logs (record-changeset-info)
   
   [hooks]
   pretxnchangegroup.a_singlehead = python:mozhghooks.single_head_per_branch.hook
   pretxnchangegroup.z_linearhistory = python:mozhghooks.pushlog.log
   
   [web]
   push_ssl = false
   allow_push = *
   templates = /repos/hg_templates
   style = gitweb_mozilla
   [extensions]
   pushlog-feed = /repos/hgpoller/pushlog-feed.py
   buglink = /repos/hgpoller/buglink.py
   hgwebjson = /repos/hgpoller/hgwebjson.py

5. /var/hg$ hg serve --webdir-conf hgweb.config

NOTE: Remember to export PYTHONPATH=/repos/hghooks/ - You would get an error like this when trying to push if not set: "abort: pretxnchangegroup.a_singlehead hook is invalid (import of "mozhghooks.single_head_per_branch" failed)". In fact, I have put it on my .bashrc

NOTE2: It is not suggested to use "allow_push = *" and "push_ssl = false" since you allow everyone who can reach your server to commit changes

What I did on the client side:

1. hg clone http://localhost:8000/af
2. make some changes on the checked out repo
3. add "default-push = http://localhost:8000/af" under [paths] in my checked out repo's .hg/hgrc file
4. hg commit -m "message"
5. hg push

To check the web interface and/or the pushlog of that repo browse to:

• http://localhost:8000/af
• http://localhost:8000/af/pushlog

My wish list:

I would like to be able to have the URL to look like:

• http://localhost:8000/l10n-central/af

so I could have things like:

• http://localhost:8000/releases/l10n-mozilla-1.9.1/af
  

but for now I believe that I am in good shape and that this improvement might actually require me to setup an apache server which I am trying to avoid.

How to setup multiple hg repositories locally without apache or lighttpd

In my previous blog post I created the following:

• a single local repository with a web interface plus pushlog
  

and in this blog post, I will show you what I did to create:

• multiple local repositories with a web interface without pushlog

NOTE: I read this page which has much more information: "Publishing Repositories with hgwebdir.cgi" even though I do not use that file at all

These are the steps I followed:

1. sudo mkdir -p /var/hg
2. sudo chown -R armenzg /var/hg
3. sudo cat <<> /var/hg/hgweb.config
   [collections]
   l10n-central/ = l10n-central/
   MARKER
4. mkdir l10n-central
5. hg init l10n-central/af
6. hg init l10n-central/de
7. hg serve --webdir-conf hgweb.config

You can now do this:

$> hg clone http://localhost:8000/af
or
$> hg clone http://localhost:8000/af

Setup your own hg local repo with pushlog

I am working on generating repackages on change and the HgLocalePoller looks for the pushlog of a locale's repository. For instance: http://hg.mozilla.org/l10n-central/de/pushlog

Since my development and testing is dependent on commit changes to the l10n repositories, I need to have commit access to generate more changes and since I am not a localizer I do not have access to the l10n repositories.

Therefore there were three options for me:

1. Gain commit access to the x-testing repository on the l10n repositories and do commits in there. Inconvenience: It is a long process to gain access.
2. Create my own user repo in hg.mozilla.org. The same inconvenience as the previous one
3. Setup my own local repo with pushlog

I was lucky enough that ted was around and he proposed me and guided me to do this last option.

Steps to create your own hg repo locally with pushlog:

1. cd /repos
   
2. hg clone http://hg.mozilla.org/users/bsmedberg_mozilla.com/hghooks/
3. hg clone http://hg.mozilla.org/users/bsmedberg_mozilla.com/hgpoller/
4. hg clone http://hg.mozilla.org/users/bsmedberg_mozilla.com/hg_templates/
   
5. mkdir test-repo
6. cd test-repo
7. hg init
8. cp ../hghooks/examples-hgrc .hg/hgrc // which basically contains the following:
   

   # the a_ and z_ prefixes are for ordering:
   # any hooks which can veto an action (require-singlehead) should be run before
   # hooks which make permanent logs (record-changeset-info)
   
   [hooks]
   pretxnchangegroup.a_singlehead = python:mozhghooks.single_head_per_branch.hook
   pretxnchangegroup.z_linearhistory = python:mozhghooks.pushlog.log
   

   
   
9. export PYTHONPATH=/repos/hghooks/ // /repos is my path, use your own
   
10. vi .hg/hgrc // Add the following with your own paths:
    

    [web]
    templates = /repos/hg_templates
    style = gitweb_mozilla
    [extensions]
    pushlog-feed = /repos/hgpoller/pushlog-feed.py
    buglink = /repos/hgpoller/buglink.py
    hgwebjson = /repos/hgpoller/hgwebjson.py

11. sudo apt-get install python-simplejson //hgwebjson needs this module
12. hg serve //To start your hg server

You can now check your own hg server with pushlog:
http://localhost:8000
and your pushlog at:
http://localhost:8000/pushlog

Remember that it is a server and you have to clone first!

1. export PYTHONPATH=/repos/hghooks/
   
2. hg clone test-repo test-repo-2
3. cd test-repo-2; echo "test data" > testfile;
   
4. hg add testfile;
   
5. hg ci -m "adding a test file";
   
6. hg push ../test-repo

Understanding the Release Engineering buildbot setup (introduction)

In Release Engineering we use buildbot (an external project written in python) to automate builds for us at certain times, do builds on change and do many different things.
With this blog post I would like to introduce you to the logic of the code we use (mozilla2/master.cfg)

In general words it works like this:

1. import modules from python, twisted, buildbot and our own custom modules buildbotcustom
2. import configuration files regarding slaves and each branch
   
3. for every branch do:
1. setup a TinderboxMailNotifier
1. if L10n is enabled for this branch setup a TinderboxMailNotifier for "Mozilla-l10n" and for "Mozilla-l10n-%locale%"
2. add the HgPoller for this branch and the associated Scheduler
3. add a Periodic scheduler if enabled for this branch
4. for every nightlyBuilder generate a Nightly scheduler
1. if L10n enabled for this branch add a DependentL10n scheduler
5. for every platform defined for this branch do:
1. create builders for dependent builds (periodic and build-on-change)
   
2. create builders for nightly builds (if it is not a debug platform)
1. if L10n enabled for this branch add a nightly builder for this platform
4. end of for every branch loop
5. import Release Automation configuration
   
6. import Mobile configuration
7. import UnitTest configuration (it is coming soon)

NOTE: I am omitting smaller details and I could be slightly wrong in some of the steps but we can go deeper into details if I do more blog posts