Showing posts with label cairo. Show all posts
Showing posts with label cairo. Show all posts

Friday, March 07, 2008

armenzg: Session 15: When things go wrong

I was trying to write cairobuild.py as a file containing steps commands which derived from ShellCommand so I could have logically related steps grouped together. For instance, "autogen.sh", "make" and "make install", since all of them have to run to count as a "good" build. Another reason was to have a summary of the tests run, instead of having a look at the STDIO to see the results of the tests.

Well, I decided to make it happen at school were we have computers (being setup by us, here and there) for our Open Source Projects, but things went wrong:

  1. I try to get buildbot and twisted on the mac mini
  2. I want to use package manager: macPorts cannot reach its rsync server; fink can reach a cvs repository to selfupdate but takes for ever
  3. I decided to get tarball and build it myself; Buildbot only works with Twisted 2.5 but I would only have 2.4 even tough I build it (sudo python ./setup.py install)
  4. I realized that buildbot got installed under /Libraries/python/2.5/site-packages (I think) and twisted too, but it seemed to don't catch it as 2.5
  5. I found through a blog post (I left the link at school) talking about it and it seemed I had to export PYTHONPATH to point at the right place
  6. Now I should had been ready, but when I tried to start my buildbot master I got this message:
    Traceback (most recent call last):
    File "/Library/Python/2.5/site-packages/twisted/application/app.py", line 379, in run
    runApp(config)
    File "/Library/Python/2.5/site-packages/twisted/scripts/twistd.py", line 23, in runApp
    _SomeApplicationRunner(config).run()
    File "/Library/Python/2.5/site-packages/twisted/application/app.py", line 157, in run
    self.application = self.createOrGetApplication()
    File "/Library/Python/2.5/site-packages/twisted/application/app.py", line 207, in createOrGetApplication
    application = getApplication(self.config, passphrase)
    --- ---
    File "/Library/Python/2.5/site-packages/twisted/application/app.py", line 218, in getApplication
    application = service.loadApplication(filename, style, passphrase)
    File "/Library/Python/2.5/site-packages/twisted/application/service.py", line 341, in loadApplication
    application = sob.loadValueFromFile(filename, 'application', passphrase)
    File "/Library/Python/2.5/site-packages/twisted/persisted/sob.py", line 215, in loadValueFromFile
    exec fileObj in d, d
    File "buildbot.tac", line 9, in
    BuildMaster(basedir, configfile).setServiceParent(application)
    File "/Library/Python/2.5/site-packages/buildbot/master.py", line 370, in __init__
    self.status = Status(self.botmaster, self.basedir)
    File "/Library/Python/2.5/site-packages/buildbot/status/builder.py", line 1779, in __init__
    assert os.path.isdir(basedir)
    exceptions.AssertionError:
And this is where I basically left it and I should find why next time I go to school or just choose one of the Linux computers and run it there, but I can't promise that everything will be easy.

CairoTests class


class CairoTests(ShellCommand):
name = "Run Cairo Tests"
warnOnFailure = True
description = ["running tests"]
descriptionDone = ["tests completed"]
command = ["make", "test"]
workdir = "cairo"

def createSummary(self, log):
passCount = 0
failCount = 0
for line in log.readlines():
if "PASS" in line:
passCount = passCount + 1
if "FAIL" in line:
failCount = failCount + 1
summary = "TestsResults:" + str(passCount) + "/" + str(failCount) + "\n"
self.addCompleteLog('summary', summary)

def evaluateCommand(self, cmd):
superResult = ShellCommand.evaluateCommand(self, cmd)
if SUCCESS != superResult:
return WARNINGS
if None != re.search('FAIL', cmd.logs['stdio'].getText()):
return WARNINGS
return SUCCESS

After this blog post I will have to change this code because I believe it does not capture properly how many tests failed or passed.
Cairo's tests have the keywords PASS, XFAIL, UNTESTED and FAIL at the end of some lines and the summary of each group of tests is after the block of tests and appears at the beginning of the line followed by a semicolon. Check the next quoted text:

TESTING svg-surface-source
Test using various surfaces as the source
svg-surface-source-image-argb32 [0]: PASS
svg-surface-source-image-argb32 [25]: PASS


<-- There are more lines but nothing relevant -->

svg-surface-source-xlib-fallback-rgb24 [0]: UNTESTED
Failed to open display: :0.0
Failed to open display: :0.0
svg-surface-source-xlib-fallback-rgb24 [25]: UNTESTED
PASS: svg-surface-source <-- This is what I call the summary of the block of tests

We also have at the end of the STDIO that shows the results of all tests and my summary should match that OR maybe I should just show that in the summary instead:

========================================================================
7 of 156 tests did not behave as expected (4 unexpected passes)
Please report to http://bugs.freedesktop.org/enter_bug.cgi?product=cairo
========================================================================
make[4]: *** [check-TESTS] Error 1
Failed tests:
ft-text-vertical-layout-type1: image test-fallback test-meta test-paginated
Failures per surface - image: 1, test-fallback: 1, test-meta: 1, test-paginated: 1.

Next releases

It seems that the Hera Try Server will be up and running something really soon and I will have to make sure that the tests are being run properly and that it meets the desires of the Mozilla community. Adam has been doing a great job and as I have heard from Shaver they will be really happy to see this working.

In my next release I will expect to have the Cairo steps for Windows and the performance tests (which I thought I could have got done for this week but did not happen) and whatever is required for me to work on the Try Server

Friday, February 29, 2008

armenzg: Sess14: Building Cairo on Mac; No problems?? No way!!

It has been quite a long time and time constraints are always flying on top of me (like most of you reading this)
Anyways, I am going to work on writing the steps to build Cairo on Mac, let's see if there are differences


NOTE: This has been my longest session when I expected to spend just an hour or two - I have found so many obstacles that I cannot recall all of them

NOTE2: No performance tests and the "make test" takes a pretty LONG TIME - More than 30 minutes (In Linux I believe it took shorter time, not sure)

Requirements

  • sudo port install git-core
  • sudo port install buildbot
svn is not in ports' repository (I think I just saw svn extensions for python and perl) so I can get if with Fink or get an installer (Subversion .dmg); I decide to go the command line way:
  • sudo port install autoconf
  • sudo port install automake
  • sudo port install libtool //I think I did: fink install libtool instead

Set-up

  • mkdir sandbox && cd sandbox
  • buildbot create-master cairomaster
  • cd cairomaster
  • I tried to checkout buildbot-configs/cairo-one-ubuntu-slave/master.cfg from my repository but svn does not checkout single files; But I placed the master.cfg to the ~/cairomaster folder
  • buildbot create-slave ./cairoslave localhost:9876 macslave slavepassword

Running

  • buildbot start cairomaster/
  • buildbot start cairoslave/
  • firefox http://localhost:8020

Conclusion

One of my longest sessions and with more problems ever!!!! Read below all my obstacles and I hope that all that I have done will help the cairo developers, in fact, cworth from the cairo channel was really helpful and he made some changes to the INSTALL file to reveal some of the things I have discovered. I love the new section called "Extremely detailed build instructions" that he has added :D

Obstacles

The autoconf and automake that come with MacOS X are not good for what I am trying to do (http://osdir.com/ml/finance.quantlib.devel/2005-09/msg00001.html ); I don't know if both or just one of them, but read below:

  • sudo port install automake // because when I try to do .autogen.sh I get the following error (even though I have libtool installed):
    NOTE2: later when trying to ./autogen.sh in cairo's folder I had a similar problem and I got libtool from Fink since MacPort was skipping since I already had libtool; After that it worked

    Can't exec "libtoolize": No such file or directory at /usr/share/autoconf/Autom4te/FileUtils.pm line 288, line 3.
    autoreconf: failed to run libtoolize: No such file or directory
  • I don't know what I did, I might have got autoconf from Fink or MacPorts // when I try to run ./autogen.sh with cairo I get this error:
    ./autogen.sh: line 177: libtoolize: command not found
  • I did not know how to configure PKG_CONFIG_PATH but now I do:
    PKG_CONFIG_PATH=/Users/armenzg/Sandbox/libs/lib/pkgconfig //since I configured pixman to build there and it created a pkgconfig folder with a .pc file
    export PKG_CONFIG_PATH // to export to the environment variables
  • After setting up the PKG_CONFIG_PATH properly and tried again to ./autogen.sh:
    configure: error: Cairo requires at least one font backend.
    Please install freetype and fontconfig, then try again:
    http://freetype.org/ http://fontconfig.org/
  • I ended up doing "sudo fink install fontconfig" because neither through Fink or MacPorts I got it working
  • Another problem, after doing a "make":
    libtool: ltconfig version `' does not match ltmain.sh version `1.3.5'
    Fatal configuration error. See the libtool docs for more information.
    make[2]: *** [libcairo_la-cairo.lo] Error 1
  • The libtool that comes with MacOS X is not the one we want (thanks cworth from #cairo channel on freenode):
    LIBTOOLIZE=glibtoolize ./autogen.sh --prefix=/Users/armenzg/Sandbox/libs
  • This last problem was found when I wrote the buildbot steps and I was trying to set environment variable PKG_CONFIG_PATH but that is not Shell Command and I found through bhearshum that I can do this:
    macFactory.addStep(step.ShellCommand, name="autogen for cairo",
    descriptionDone="autogen for cairo",
    command=["./autogen.sh","--prefix="+mac_build_prefix+"/lib"],
    workdir="cairo",
    env={"PKG_CONFIG_PATH": '%s/lib/lib/pkgconfig' % mac_build_prefix,
    "LIBTOOLIZE": 'glibtoolize'})

Tuesday, February 12, 2008

armenzg: Sess13: Buildbot with Cairo

In this post you will see that I have set up a master.cfg file to build cairo and run the tests BUT not the performance tests

My results:

What the buildbot steps do is basically this:

  • Remove folders "pixman" and "cairo"
  • Checkout from GIT repository "pixman" and "cairo"
  • Build pixman
    • ./autogen.sh
    • ./configure --prefix=/home/armen/lib //This is to avoid to do "sudo make install"
    • make
    • make install
  • Build Cairo
    • ./autogen.sh
    • ./configure --prefix=/home/armen/lib
    • make
    • make install
  • Run Cairo's Tests
    • make test
    • cd test && make html //This generates html file that displays images, all together is around 35MB
  • Run Cairo's performance tests
    • TO BE DONE

QUESTIONS:

  • How do I pack the results and provide it? Upon request? After each build?
  • PKG_CONFIG_PATH tells pkg-config where to look for information about packages and we use pkg-config to locate things like glib and cairo.
    My question is when am I supposed to run export PKG_CONFIG_PATH=/home/armen/lib?? //I don't have it in my master.cfg
  • I still have to run the performance tests and see what happens

TO REMEMBER:



EXTRA, BUT DO NOT READ:

FROM THIS POINT ON, these steps are my first attempt and you should not waste your time reading it, but it is good for learning purposes


This is a FRESH manual build of Cairo, the steps to set up the environment are ommited.
I have a problem/question relating that I have to raise privileges to do "make install", will this be OK with buildbot? Is there a way around?

  • mkdir cairo & cd cairo
  • git clone git://git.cairographics.org/git/pixman
  • git clone git://git.cairographics.org/git/cairo
  • cd pixman && ./autogen.sh
  • ./configure
  • make
  • make install //I have a problem can I set up buildbot to use SUDO??
    make[2]: Entering directory `/home/armen/sandbox/cairo2/pixman/pixman'
    test -z "/usr/local/lib" || /bin/mkdir -p "/usr/local/lib"
    /bin/bash ../libtool --mode=install /usr/bin/install -c 'libpixman-1.la' '/usr/local/lib/libpixman-1.la'
    /usr/bin/install -c .libs/libpixman-1.so.0.9.6 /usr/local/lib/libpixman-1.so.0.9.6
    /usr/bin/install: cannot remove `/usr/local/lib/libpixman-1.so.0.9.6': Permission denied
    make[2]: *** [install-libLTLIBRARIES] Error 1

  • sudo make install //I had to use SUDO
  • whereis libpixman-1
  • cd ../cairo && ./autogen.sh
  • ./configure
  • make
  • make install //the same problem with SUDO
    make[2]: Entering directory `/home/armen/sandbox/cairo2/cairo/src'
    test -z "/usr/local/lib" || /bin/mkdir -p "/usr/local/lib"
    /bin/bash ../libtool --mode=install /usr/bin/install -c 'libcairo.la' '/usr/local/lib/libcairo.la'
    /usr/bin/install -c .libs/libcairo.so.2.14.0 /usr/local/lib/libcairo.so.2.14.0
    /usr/bin/install: cannot create regular file `/usr/local/lib/libcairo.so.2.14.0': Permission denied
  • sudo make install

  • make test //takes long time
  • cd test && make html

Tuesday, January 22, 2008

armenzg: Sess7 - Build Cairo

Another project that is really similar to the one I am doing with Mozilla is to set up a Buildbot Farm to build the latest Cairo code (Mozilla relies on Cairo), to run unit testing tests and to run performance tests.
So far what I have done is to try to build manually Cairo in Linux Ubuntu 7.1

I can say that I did not find instructions on how to build Cairo over the internet and neither on Cairo's website (the closest thing is http://cairographics.org/download/ and something about building cairomm but I can't find the URL anymore). Maybe there is some info in here, http://lists.cairographics.org/archives/cairo/

NOTE: Candy bar at the bottom of this post

Anyways, these are the steps I manage to do by the help of behdad, ssp (from freenode #cairo) and cesar (from #seneca)
NOTE = I don't have most of the concepts behind building things on Linux, which might be obviuos to a lot of Linux developers

Get the source code:
  • sudo apt-get install git-core //To get the GIT Version Control
  • git clone git://git.cairographics.org/git/cairo //To get the latest source code of Cairo - Checkout 2m43s
  • git clone git://git.cairographics.org/git/pixman //Cairo cannot be built without this project installed - Checkout 3sec

Some instructions that will apply for Builbot configuration (I think)
  • makedir ~/buildslave/base TO BE EXPLAINED
  • PKG_CONFIG_PATH=~/buildslave/base/
NOTE: I read this in an email I got but I am not sure why or if I did it right

Let's build pixman before cairo
  • cd to where I have pixman
  • I tried to run ./configure as it said in pixman's INSTALL file but nothing
  • I tired to run ./autogen.sh but it said that could not find autoreconfig or something like that
  • The next three steps were needed for me to be able to run ./autogen.sh
  • sudo apt-get install autoconf //as suggested in IRC
  • sudo apt-get install automake //from errors in screen
  • sudo apt-get install libtool //as suggested in IRC but could be induced from errors
  • ./autogen.sh //FINALLY
  • ./configure
  • make
  • sudo make install
  • To see if you have the pixman library do this:
    whereis libpixman-1 //I think that it should work even before we run sudo ldconfig

Build Cairo
  • Then CD to where the Cairo source code is
  • ./autogen.sh
  • make
  • sudo make install

Let's run some tests and others
  • make test //Allmost all tests out of 147 have failed
  • One of the failed tests error: "/home/armen/sandbox/cairo/cairo/test/.libs/lt-xlib-surface: error while loading shared libraries: libpixman-1.so.0: cannot open shared object file: No such file or directory"
  • cd test && make html //"Created index.html" - Just to see what will I get
  • sudo ldconfig // Suggested by Cesar - link to explanation
  • QUESTION: What is the difference if I do "make test" inside of test or one level above?
  • make test // A lot of graphics going on and 32 of 147 tests failed
  • make html //"Created index.html" - Just to see what will I get

The Candy bar
I have packed the index.html file with the png and the log files into a tar.gz file so you can see the results of what the tests of Cairo look like (Open index.html to see it)
I have packed the index.html file with the png and the log files into a tar.gz file so you can see the results of what the tests of Cairo look like.
NOTE: I have removed 300 objects from it to reduce the file size from 50 something MB to 9 MB.
NOTE2: You still have another 5000+ objects that I have not removed