Sebastian Rittau's Blog

Friday
Jul 14 2006

XML Test Runner for PyUnit

Python, Programming

JUnit features an XML Test Runner. This means that the result of a test run is not written to stdout, but instead written into an XML file. This XML file can then be automatically processed. This is for example useful for fully automated builds using software like CruiseControl. The XML output is easily converted into an XHTML file for easy human reading.

Until now PyUnit was lacking an XML Test Runner. Since I needed one for one of my projects that used CruiseControl, I wrote one. It is available for download here. Since this is an extension to a unit testing framework, the classes are of course fully tested. :)

If you have any suggestions or criticism, please let me know. I'm a nitpicker myself, so even if you have some small suggestions about coding style or improvement hints, I would like to hear about them.

Update: I have now submitted this patch to Python's patch tracker, ticket number 1522704.

Update^2: I created a page with more information about PyUnit and CruiseControl integration, including sample configuration files and scripts.

Update^3: I updated the XmlTestRunner after input from Mirko Friedenhagen: It now recovers gracefully from unit tests overriding sys.stdout and sys.stderr, the XmlTestRunner returns the TestResult instance instead of a boolean value, and you can now stream multiple test suite results into a single XML file, since the XML header is not written to file streams by default.

Comments

Sample CC-Configuration

by Mirko

Hello,

thanks for this, maybe you could share a sample CC-configuration-file as well?

Regards

Mirko

Thanks!

by Micah

Hi, I found your test runner to be quite useful. It took me a bit to figure it out, but all-in-all it was pretty easy.

Now I just have to figure out the best way to integrate it with CC.

Anyways, thanks!

Patch to make it run with python 2.3

by Lars 

--- xmlrunner.py     2007-01-30 10:36:02.000000000 +0100
+++ xmlrunner.py     2007-01-30 11:06:25.000000000 +0100
@@ -25,15 +25,18 @@
     Used by _XmlTestResult."""
     def __init__(self, test, time):
-        (self._class, self._method) = test.id().rsplit(".", 1)
+        def mysplit(fullstring, substring) :
+            pos = fullstring.rfind(substring)
+            return (fullstring[:pos], fullstring[(pos+1):])
+
+        (self._class, self._method) = mysplit(test.id(), ".")
         self._time = time
         self._error = None
         self._failure = None
-    @staticmethod
     def create_success(test, time):
         """Create a _TestInfo instance for a successful test."""
         return _TestInfo(test, time)
+    create_success = staticmethod(create_success)
-    @staticmethod
     def create_failure(test, time, failure):
         """Create a _TestInfo instance for a failed test."""
@@ -41,6 +44,6 @@
         info._failure = failure
         return info
+    create_failure = staticmethod(create_failure)
-    @staticmethod
     def create_error(test, time, error):
         """Create a _TestInfo instance for an erroneous test."""
@@ -48,4 +51,6 @@
         info._error = error
         return info
+    create_error = staticmethod(create_error)
+
         def print_report(self, stream):

Sorry ...

by Lars

.. about the mess in the previous message. Feel free to remove it or fix and make it available somehow.

by Sebastian Rittau

Thank you for the patch! I've fixed the formatting. Currently my blog doesn't support direct input of code blocks. (Something I should fix at some point.)

Where's a good XSL file to convert the output into a report?

by Phlip

Where's an XSL that works on the output?

I Google for things called Junit.*xsl, and they claim they will work, but they don't.

The ones I found have lines like these, indicating they might match the TEST-unittest.TestSuite.xml file's nodes:

<xsl:template match="testcase" mode="print.test">

I could just be calling them wrong; this emits nothing:

xsltproc report/junit-noframes.xsl TEST-unittest.TestSuite.xml

I apologize to everyone for having the temerity to unit test Python without learning Java & JUnit first, but ... you had to be there!