[TIP] Result protocol / problems being solved
fuzzyman at voidspace.org.uk
Tue Apr 14 03:39:33 PDT 2009
Robert Collins wrote:
> On Tue, 2009-04-14 at 11:13 +0100, Michael Foord wrote:
>> It looks good to me. I'm a new committer so I still need the nod from
>> another developer. Once I have pushed out fresh Mock and ConfigObj
>> releases I'll return to unittest issues.
>> I'd like to just apply:
>> * startTestRun / stopTestRun
>> * a keyword argument to main (TestProgram) *not* to call sys.exit
>> * addCleanUp (with the fix to use while / pop instead of iterating
>> through reversed and with a final resolution to the order debate)
> Cool. I'm still strongly on 'run after', but honestly, make a decision
> and land it.
I've made a decision, have justification and also a plan to mitigate the
difficulties for the 'other camp'. I'd like to keep debate on the
tracker though so lets not dwell on it here - as you say, it just needs
to go in.
>> I'd also like to review the 'removeTest' method patch. I like the
>> functionality but am not sure about the patch.
> I haven't looked at this - I have no strong opinion about it. Would it
> be useful for me to review it?
That would be really helpful as I think it is a reasonable feature request.
>> After this I'd like to do a proper implementation of the discovery
>> stuff. There is still one decision that needs to be made (not discussed
>> so far) - should the top level directory be treated as a package? At the
>> moment it is treated as a directory and all modules loaded separately.
>> It should probably be checked for an __init__.py first and treated as a
>> package if it exists.
> Perhaps we can recast it slightly - if it is 'discover from a name' then
> the question of top level being directory or package or not is moot: it
> import name
> start_dirs = name.__path__
> (loop here)
> That seems to remove the need to take such a decision at all.
That specifically requires a package name as input rather than a
filesystem location - so that is making a decision!
It does allow for test packages that extend themselves through __path__
manipulation. I think I'd like to see an implementation of this
I think I'm now marginally in favour of providing the implementation of
this in loadTestsFromPackage - with the load_tests protocol being in
loadTestsFromModule as you showed previously.
>> After that I would like to turn unittest and its tests into a package.
>> I'm hoping to do all of this in time to get it into 2.7.
> I'd like to cleanup the skip support somewhat, the classsuiteclass thing
> is odd - do you (or anyone here) know what the motivation behind it is?
> AFAICT it is just to report 'a skipped module' as one skip rather than
> one-per-test (which is IMO a misfeature, as test totals won't add up
> properly then).
The skipped stuff was added before I got involved and I don't recall
much discussion on the feature. A tracker item will probably help here.
>> One other issue - at the moment the only way to use a custom TestResult
>> is to to override the '_makeResult' method (off the top of my head - may
>> not be quite the correct method name) of TestRunners. Would it make
>> sense for the runTests method to take an optional result argument?
> Well, TextTestRunner requires a TextTestResult, which is a little more
> than a TestResult.
> I'd like to see a runner that uses only the methods the top level
> TestResult interface provides, and have more of the output live on the
> TestResult. This would make suppling a runner, or a runner factory more
> useful and flexible.
OK, lets take that offline until the other issues are dealt with. As I
said, I have two releases of my own stuff pending that I need to get out
of the way.
More information about the testing-in-python