[TIP] Functional Test Runner for Command Line Tools?

C. Titus Brown ctb at msu.edu
Wed Apr 15 18:46:58 PDT 2009


On Wed, Apr 15, 2009 at 03:37:07PM -0500, Augie Fackler wrote:
-> On Apr 13, 2009, at 11:15 PM, Noah Gift wrote:
-> > In looking at http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy
-> > I don't see something that automates writing functional tests for
-> > command line tools.  Hopefully, I am wrong, but I am looking for
-> > something that will fully exercise a command line tool easily.  How is
-> > everyone else testing their command line tools?
-> 
-> I'd highly recommend you look at the Mercurial testsuite - they store  
-> a shell script and expected output and then print a diff of any  
-> changes, and store the new output in a file next to the expected  
-> output. I've found it very convenient.

Trivial code example from the figleaf test suite:

the .good files in this directory,

http://github.com/ctb/figleaf/tree/d11d2ee0ce70427b229af15409bff2ab23e090d2/tests/test_bin_progs

like 'test-cover.py.report.good',

http://github.com/ctb/figleaf/blob/d11d2ee0ce70427b229af15409bff2ab23e090d2/tests/test_bin_progs/tst-cover.py.report.good

are used by test functions, e.g. 'test_figleaf_report' in 

http://github.com/ctb/figleaf/blob/d11d2ee0ce70427b229af15409bff2ab23e090d2/tests/test_bin_progs/__init__.py

to do comparisons with "known good" output.

This kind of thing is what I personally call a regression test[0]
because it only detects differences in output between then and now,
rather than asserting that something is actually correct.

cheers,
--titus
--
C. Titus Brown, ctb at msu.edu

[0] Your terminology may vary.  Terminology not valid outside of Dr.
Brown's Reality Distortion Field.



More information about the testing-in-python mailing list