No subject


Sun Sep 27 01:15:05 PDT 2009


rather than just state. The "statist" view is that it is sufficient to
put the SUT into a specific state, exercise it, and verify that it is
in the expected state at the end of the test. The "behaviorist" view
is that we should specify not only the start and end state of an
object but the calls it makes to its dependencies. That is, we should
specify the details of the calls to the "outgoing interfaces" of the
SUT. I call these "outgoing calls" the indirect outputs of the SUT
because they are outputs just like the values returned by functions
except that we must use special measures to trap them because they
don't come directly back to the client or test.

The behaviorist school of thought is sometimes called behavior-driven
development. It is evidenced by the copious use of Mock Objects or
Test Spys throughout the tests. It does a better job of testing each
unit of software in isolation at a possible cost of more difficult
refactoring.
"""

I believe this is what happened to you.

So the first thing to note is that you may want to try to loosen up
your test expectations a bit and verify only state instead of whole
behavior, unless it really makes sense (e.g. when order of actions
really matter).

Moreover, in a "imperative-heavy environment" where your program does
a lot of things I think it's better *not* to rely on mocks unless you
really need to. For example, instead of mocking filesystem, use a
temporary directory and shutil.rmtree it once you're done. Instead of
mocking os.remove, allow the application to remove the file and check
if the file still exists afterwards. That goes back to the point that
you don't need to control use of those specific functions - you're
only interested in the outcome (i.e. the end state).

Another thing that I noticed about your library is that you lack
integration tests - tests that would test your entire codebase on a
real system. While unit tests are useful during development, they in
fact may constitute a problem during large refactorings. In that cases
I would advise to simply write new unit tests for the new interface
and rely on functional, higher-level tests to catch regressions. If
you're changing your interfaces it doesn't make much sense to keep old
unit tests (which are coupled to those interfaces) around anymore. But
you need the functional tests first to do that!

Yet another thing that I noticed is that you use your own test runner.
Using something like nose (http://code.google.com/p/python-nose/)
could remove first part of your test/scaffold.py.

Finally, the following remark may sound obvious, but it's always good
to keep in mind: complexity of the code is reflected in the complexity
of the tests. More functional style you use in your code the more
straightforward your tests are. I don't know the code base of
python-daemon well, but I can imagine there's a lot of imperative
coding going on - you check permissions, you create files, you spawn
threads, etc. One thing you may try is look for places in code which
could be made more functional - instead of modifying state they should
return values.

As an example of this technique I've extracted a message formatting
code to a separate method, see:

http://bazaar.launchpad.net/~ruby/+junk/python-daemon/revision/185

Now you can test "_usage_message" separately, without the need for any mocks.

To help you test classes that rely on other components you may do one
of two things:
 * use dependency injection - pass all required objects to __init__
explicitly (since you're writing a library this is also good for
extensibility),
 * extract that code out from __init__ into another method, which you
can then easily shadow in a mocking subclass.
Basically the idea is to make mocking code as simple as possible. In
most cases a simple stub which will ignore all calls to it will do.

I hope that will help you make your test suite better and of course
feel free to ask more questions. :-)

Cheers,
mk



More information about the testing-in-python mailing list