[TIP] Functional Test Runner for Command Line Tools?
C. Titus Brown
ctb at msu.edu
Wed Apr 15 18:46:58 PDT 2009
On Wed, Apr 15, 2009 at 03:37:07PM -0500, Augie Fackler wrote:
-> On Apr 13, 2009, at 11:15 PM, Noah Gift wrote:
-> > In looking at http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy
-> > I don't see something that automates writing functional tests for
-> > command line tools. Hopefully, I am wrong, but I am looking for
-> > something that will fully exercise a command line tool easily. How is
-> > everyone else testing their command line tools?
->
-> I'd highly recommend you look at the Mercurial testsuite - they store
-> a shell script and expected output and then print a diff of any
-> changes, and store the new output in a file next to the expected
-> output. I've found it very convenient.
Trivial code example from the figleaf test suite:
the .good files in this directory,
http://github.com/ctb/figleaf/tree/d11d2ee0ce70427b229af15409bff2ab23e090d2/tests/test_bin_progs
like 'test-cover.py.report.good',
http://github.com/ctb/figleaf/blob/d11d2ee0ce70427b229af15409bff2ab23e090d2/tests/test_bin_progs/tst-cover.py.report.good
are used by test functions, e.g. 'test_figleaf_report' in
http://github.com/ctb/figleaf/blob/d11d2ee0ce70427b229af15409bff2ab23e090d2/tests/test_bin_progs/__init__.py
to do comparisons with "known good" output.
This kind of thing is what I personally call a regression test[0]
because it only detects differences in output between then and now,
rather than asserting that something is actually correct.
cheers,
--titus
--
C. Titus Brown, ctb at msu.edu
[0] Your terminology may vary. Terminology not valid outside of Dr.
Brown's Reality Distortion Field.
More information about the testing-in-python
mailing list