Tuesday, September 13, 2016

Increasing test coverage

Last quarter I spent some time increasing mozci's test coverage. Here are some notes I took to help me remember in the future how to do it.

Here's some of what I did:
  • Read Python's page about increasing test coverage
    • I wanted to learn what core Python recommends
    • Tthey recommend is using coverage.py
  • Quick start with coverage.py
    • "coverage run --source=mozci -m py.test test" to gather data
    • "coverage html" to generate an html report
    • "/path/to/firefox firefox htmlcov/index.html" to see the report
  • NOTE: We have coverage reports from automation in coveralls.io
  • If you find code that needs to be ignored, read this.
    • Use "# pragma: no cover" in specific lines
    • You can also create rules of exclusion
  • Once you get closer to 100% you might want to consider to increase branch coverage instead of line coverage
    • Read more in here.
  • Once you pick a module to increase coverage
    • Keep making changes until you run "coverage run" and "coverage html".
    • Reload the html page to see the new results
After some work on this, I realized that my preferred place to improve tests is focusing on the simplest unit tests. I say this since integration tests do require proper work and thinking how to properly test them rather than *just* increasing coverage for the sake of it.

Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.