[TIP] Ideology

[Douglas Philips]

On or about 2009 Apr 21, at 3:27 PM, Jesse Noller indited:
> On Tue, Apr 21, 2009 at 3:13 PM, Douglas Philips wrote:
>> Right. If your tests are not, de facto, testing your framework, your
>> framework has bloat that should be eliminated, it isn't carrying its
>> own weight.
>
> Hooey. Your tests are obviously functional tests for your framework,
> however as someone who *writes* the framework, I need unit tests to
> make sure I didn't regress the damned thing before I foist it on the
> unsuspecting world.

Then you just end up woodshedding general abstractions that your  
customers "might" need without knowing if they do need them or not.  
Been down that path already.


>> This is the inflection point where unit testing diverges from, say,
>> device testing (the kind of stuff I'm fortunately paid to do). And so
>> it begs the question of whether unit tests and nose and py.test are
>> really all the same, not to mention hardware testing, system/
>> functional testing... The more complex the "system under test", the
>> more involved the testing framework seems to get...
>
> <soapbox>
> A good executor doesn't care what the classification of a given test
> is (functional, unit, regression, mymom). A good executor only finds
> tests and runs them.

Well, you can define it that way, is that what nose does?
I notice that py.test has a lot of functionality I do -not- need for  
doing distributed test execution.
Nose and py.test have looks of "cool unit test" junk that I just  
simply don't need doing my device testing.
And I have a bunch of functionality in my test framework that unit- 
tests don't need.
If there were a nice composible framework, I could build just what I  
need and so could you.
I haven't looked at nose, but I did look a py.test and OMFG there are  
tomes of code in there I will never use... how could nose be any  
better when it is solving lots of problems I. do. not. have.


> Ultimately, unittest is geared towards unit testing (duh) - but I (and
> others) use it as a perfectly good working base for *any* sort of
> test. I've seen it used for drive testing, filesystem testing, etc.

Yes, that is exactly what I've done too, but I have to have my own  
version of it because it is not extensible in the directions I needed  
and there is stuff in it that I don't need and should have already  
removed. We all know how bad it is to have dead code bloating  
around... But how much do normal unit-tests need my kind of test  
selection and filtering and ... probably all features that most unit  
tests would balk at putting into a "general" framework, and rightly so.

-Doug