Project definition: SETA re-write

As an attempt to attract candidates to GSoC I wanted to make sure that the possible projects were achievable rather than lead them on a path of pain and struggle. It also helps me picture the order on which it makes more sense to accomplish.

It was also a good exercise for students to have to read and ask questions about what was not clear and give lots to read about the project.

I want to share this and another project definition in case it is useful for others.

----------------------------------

We want to rewrite SETA to be easy to deploy through Heroku and to support TaskCluster (our new continuous integration system) [0].

Please read carefully this document before starting to ask questions. There is high interest in this project and it is burdensome to have to re-explain it to every new prospective student.

Main mentor: armenzg (#ateam)

Co-mentor: jmaher (#ateam)

Please read jmaher’s blog post carefully [1] before reading anymore.

Now that you have read jmaher’s blog post, I will briefly go into some specifics.

SETA reduces the number of jobs that get scheduled on a developer’s push.

A job is every single letter you see on Treeherder. For every developer’s push there is a number of these jobs scheduled.

On every push, Buildbot [6] decides what to schedule depending on the data that it fetched from SETA [7].

The purpose of this project is two-fold:

1. Write SETA as an independent project that is:
1. maintainable
2. more reliable
3. automatically deployed through Heroku app
2. Support TaskCluster, our new CI (continuous integration system)

NOTE: The current code of SETA [2] lives within a repository called ouija.

Ouija does the following for SETA:

1. It has a cronjob which kicks in every 12 hours to scrape information about jobs from every push
2. It takes the information about jobs (which it grabs from Treeherder) into a database

SETA then goes a queries the database to determine which jobs should be scheduled. SETA chooses jobs that are good at reporting issues introduced by developers. SETA has its own set of tables and adds the data there for quick reference.

Involved pieces for this project:

1. Get familiar with deploying apps and using databases in Heroku
2. Host SETA in Heroku instead of http://alertmanager.allizom.org/seta.html
1. https://bugzilla.mozilla.org/show_bug.cgi?id=1253020
3. Teach SETA about TaskCluster
1. https://bugzilla.mozilla.org/show_bug.cgi?id=1243123
4. Change the gecko decision task to reliably use SETA [5][6]
1. If the SETA service is not available we should fall back to run all tasks/jobs
5. Document how SETA works and auto-deployments of docs and Heroku
1. Write automatically generated documentation
2. Add auto-deployments to Heroku and readthedocs
6. Add tests for SETA
1. Add tox/travis support for tests and flake8
7. Re-write SETA using ActiveData [3] instead of using data collected by Ouija
1. https://bugzilla.mozilla.org/show_bug.cgi?id=1253028
8. Make the current CI (Buildbot) use the new SETA Heroku service
1. https://bugzilla.mozilla.org/show_bug.cgi?id=1252568
9. Create SETA data for per test information instead of per job information (stretch goal)
1. On Treeherder we have jobs that contain tests
2. Tests re-order between those different chunks
3. We want to run jobs at a per-directory level or per-manifest
10. Add priorities into SETA data (stretch goal)
1. Priority 1 gets every time
2. Priority 2 gets triggered on Y push

[0] http://docs.taskcluster.net/

[1] https://elvis314.wordpress.com/tag/seta/

[2] https://github.com/dminor/ouija/blob/master/tools/seta.py

[3] http://activedata.allizom.org/tools/query.html

[4] https://bugzilla.mozilla.org/show_bug.cgi?id=1243123

[5] https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=gecko

[6] testing/taskcluster/mach_commands.py#l280

[7] http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla-tests/config_seta.py

[8] http://alertmanager.allizom.org/seta.html

---

This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.