[TIP] Interface vs Implementation Testing
holger krekel
holger at merlinux.de
Wed Apr 25 02:05:02 PDT 2007
Hi Essien, Max, hi all,
On Tue, Apr 24, 2007 at 17:02 +0100, Essien Ita Essien wrote:
> Max Ischenko wrote:
> > On 4/24/07, *Titus Brown* <titus at caltech.edu <mailto:titus at caltech.edu>>
> > wrote:
> >
> > -> I coded a huge test suite once that primarily unit tested
> > -> implementation. Since there were many tests, any time I wanted to
> > -> change the way the app was implemented (faster algorithm here, don't
> > -> need this class anymore, more methods now because of feature X) I
> > -> usually ended up spending a good deal of time updating and deleting
> > -> tests. This just seemed unecessary to me and deterred me from
> > relying
> > -> on the test suite as an indication of successful functionality.
> >
> > So, isn't this kind of an argument for functional and regression tests
> > only?
> >
> > (I'm honestly curious. I haven't found much of a need for unit
> > tests in
> > my own code.)
> I've been mostly silent and stealing ideas and lessons from others
> so i guess i'd better contribute back a thought or two here ;-)
>
> For me... I don't even do much of what you call functional testing, but
> what i find invaluable are those unit tests. How I normally develop is
> in quick spurts of small ideas that i'll eventually put together into a
> whole. for instance, I'm gonna need to read a files over the network, so
> i quickly code a function that does just that, i'm going to need to
> steganograph streaming data from various sources and stuff to files, so
> i quicly code functions needed to do just that.
>
> The problem is that without the whole application, you really can't test
> these little bits and pieces. But with UNIT tests... the whole ball game
> changes. You are no more writing a bit application, but a small
> application that does just A for just B, and you can setup the small
> environment in your unittest, needed to get A or B to just work. Once
> its workign... you can safely plunk that functionality int a module
> and be rest assured that when you actually call it in your big
> application... its gonna do just what you intended it to do.
>
> When its time to do the putting together... if all your parts are
> working as expected in other small small applications (aka. Unit Tests).
> You just go to sleep :)
nicely put. In fact, I guess for complex programs involving many possible
combinations of user input and involved code/component interactions
unit tests are kind of neccessary to get to a maintainable and consistent
code base. For PyPy (being a full Python Interpreter and a Translation framework
to various backends) there would have been no way to make progress without unit-testing.
However, we used large functional tests (such as translating the whole Python
Interpreter to C) to try to identify some problems for which we then added unit-tests
to prove our theory about the problem. Then we fixed it.
Then again, the lines blurry - our unit-tests are often not strict unit-tests
but rather small functional tests. To me, there is a grey area and actually
i don't care too much. What matters to me regarding a test is:
debug information/hints:
will a failure and the related information/debugging facility
allow me to identify the underlying problem quickly?
failure-to-problem-relation:
are the tests fine-grained enough that i get one failure per problem
or do i rather get one failure, but 3 independent problems are underlying it?
speed:
can i run a significant fraction of tests speedily from my development machine?
i guess that strict unit-tests usually score quite well on these
criteria, but some functional tests can qualify as well IMO.
holger
More information about the testing-in-python
mailing list