[TIP] Functional Test Runner for Command Line Tools?

C. Titus Brown ctb at msu.edu
Wed Apr 15 18:46:58 PDT 2009

On Wed, Apr 15, 2009 at 03:37:07PM -0500, Augie Fackler wrote:
-> On Apr 13, 2009, at 11:15 PM, Noah Gift wrote:
-> > In looking at http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy
-> > I don't see something that automates writing functional tests for
-> > command line tools.  Hopefully, I am wrong, but I am looking for
-> > something that will fully exercise a command line tool easily.  How is
-> > everyone else testing their command line tools?
-> I'd highly recommend you look at the Mercurial testsuite - they store  
-> a shell script and expected output and then print a diff of any  
-> changes, and store the new output in a file next to the expected  
-> output. I've found it very convenient.

Trivial code example from the figleaf test suite:

the .good files in this directory,


like 'test-cover.py.report.good',


are used by test functions, e.g. 'test_figleaf_report' in 


to do comparisons with "known good" output.

This kind of thing is what I personally call a regression test[0]
because it only detects differences in output between then and now,
rather than asserting that something is actually correct.

C. Titus Brown, ctb at msu.edu

[0] Your terminology may vary.  Terminology not valid outside of Dr.
Brown's Reality Distortion Field.

More information about the testing-in-python mailing list