We're mainly focused on making it easier for developers and allow for further flexibility.
We will initially focus on the testing side of the automation and make ground work for other further improvements down the line.
The set of changes discussed for this quarter are:
- Move remaining set of configs to the tree - bug 1067535
- This makes it easier to test harness changes on try
- Read more information from the in-tree configs - bug 1070041
- This increases the number of harness parameters we can control from the tree
- Use structured output parsing instead of regular where it applies - bug 1068153
- This is part of a larger goal where we make test reporting more reliable, easy to consume and less burdening on infrastructure
- It's to establish a uniform criteria for setting a job status based on a test result that depends on structured log data (json) rather than regex-based output parsing
- "How does a test turn a job red or orange?"
- We will then have a simple answer that is that same for all test harnesses
- Mozharness try support - bug 791924
- This will allow us to lock which repo and revision of mozharnes is checked out
- This isolates mozharness changes to a single commit in the tree
- This give us try support for user repos (freedom to experiment with mozharness on try)
Even though we feel the pain of #4, we decided that the value gained for developers through #1 & #2 gave us immediate value while for #4 we know our painful workarounds.
I don't know if we'll complete #4 in this quarter, however, we are committed to the first three.
If you want to contribute to the longer term vision on that proposal please let me know.
In the following weeks we will have more updates with regards to implementation details.
Stay tuned!
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
No comments:
Post a Comment