We're mainly focused on making it easier for developers and allow for further flexibility.
We will initially focus on the testing side of the automation and make ground work for other further improvements down the line.
The set of changes discussed for this quarter are:
- Move remaining set of configs to the tree - bug 1067535
- This makes it easier to test harness changes on try
- This increases the number of harness parameters we can control from the tree
- This is part of a larger goal where we make test reporting more reliable, easy to consume and less burdening on infrastructure
- It's to establish a uniform criteria for setting a job status based on a test result that depends on structured log data (json) rather than regex-based output parsing
- "How does a test turn a job red or orange?"
- We will then have a simple answer that is that same for all test harnesses
- This will allow us to lock which repo and revision of mozharnes is checked out
- This isolates mozharness changes to a single commit in the tree
- This give us try support for user repos (freedom to experiment with mozharness on try)
Even though we feel the pain of #4, we decided that the value gained for developers through #1 & #2 gave us immediate value while for #4 we know our painful workarounds.
I don't know if we'll complete #4 in this quarter, however, we are committed to the first three.
If you want to contribute to the longer term vision on that proposal please let me know.
In the following weeks we will have more updates with regards to implementation details.
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.