Sunday, January 13, 2008

Sess5 - xpcshell-based unit tests - Tunits (aka "make check")

First of all, I have build a non-static build of firefox without --disable-tests

Running xpcshell-based unit tests (reading from Writing_xpcshell-based_unit_tests )
  • To run one test example, go to OBJDIR and run:
    make -C tools/test-harness/xpcshell-simple/example check
  • If you just want to run ONE MODULE, cd to the module or component under the OBJDIR folder like ($OBJDIR)/browser and run "make check" to run all tests of that module
  • To run all xpcshell based unit tests do:
    make check //which took around 4 mins in my laptop
    which runs all the tests in ($OBJDIR)/_tests/, inside there we have the "xpcshell-simple" folder with all the xpcshell-based unit tests
NOTE: If you want to the results of a "make check" have a look at this file (makeCheckResults.txt)

The xpcshell-simple folder, who is there and how does it look like
  • In that folder we have more folders that most of them have the prefix "test" (test_autocomplete, test_browser_feeds, test_intl_uconv and others...)
  • Inside each of these folders you will find out a "unit" folder or a "test" folder (which also contains one level down a "unit" folder)
  • Each "unit" folder contains javascript files that are the actual tests.
  • You can also realize that the tests can have this format, test_.js, which is normally (if not mistaken) the number of the associated bug being tested
  • Another thing is that you get the result of the test in a log file in the same place where a xpcshell test is with a file name like this: test_.js.log. These logs look like this:
    *** test pending
    *** test pending
    *** test finished
    *** running event loop
    *** test finished
    *** exiting
    *** PASS ***

    which are also the messages that you get when doing "make check"

  • To read more about how to write a test read this (Writing_xpcshell-based_unit_tests)


More info on how things get generated
When you do a "make -f client.mk build", the make arrives to the "Makefiles.in" and read something like this (modules/libpref/test/Makefile.in ):
  46 MODULE          = test_libpref
47
48 XPCSHELL_TESTS = unit
Therefore it creates a folder "test_libref" in the "xpcshell-simple" folder (under (OBJDIR)/_tests) and inside the "test_libref" folder it creates a folder "unit" with the exact content on the source (source/modules/libpref/test/ ), which is a bunch of tests written on javascript.

When we run "make check" from the ($OBJDIR), the make goes through each folder looking (I think) for all the Makefile.in files that have XPCSHELL_TESTS defined and runs the tests that are in ($OBJDIR)/_tests/xpcshell-simple/MODULE/XPCSHELL_TESTS

It is not always easy to map back to the original file by following the folder structure in the "_tests" folder, i. e. embed/tests/unit which doesn't map to source/embed/tests/unit but it maps back to source/embedding/tests/unit/

The best way is to look for one of the Javascript tests inside of the "unit" folder, i.e. the file _tests/xpcshell-simple/embed/tests/unit/test_wwauthpromptfactory.js by doing a search for file "test_wwauthpromptfactory.js" we find it in MXR by searching for the filename

3 comments:

  1. Hi,
    Is it possible to not to get any log out put with make xpcshell-tests command? I'm trying to run some xpcshell tests in thunderbird with this command and it neither generates log file nor console output.

    make xpcshell-tests TEST_PATH=mozilla/_tests/xpcshell/calendar/test/test_mytest.js

    ReplyDelete
  2. Hi,
    Things have changed a lot since this post.
    You probably will have better luck reading the documentation:
    https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/XPCShell

    Also feel free to jump in the #ateam IRC channel and we can help you there. I hope!

    ReplyDelete
  3. yes, There was a lot of changes. Thing is I could run the tests with that command. Problem with the logging. No log file could be found with that command. I couldn't use 'mach' neither because the platform incompatibility. It was disabled because I don't currently have a B2G/Firefox/Android build. But I think 'make' should log still to a file. I'll contact #ateam. Btw, thank you for the comment :)

    ReplyDelete