[TIP] test results tracking, etc...

[Douglas Philips]

On or about 2009 Apr 5, at 5:01 PM, Robert Collins indited:
> Subunit could be described as a xUnit structured TAP, its why I
> mentioned it ...
>
> The subprocess aspect is overemphasised in the docs, I'm filing a bug
> right after this mail about that.

Thanks, does that mean it can be used without subprocesses getting  
involved? I admit that I pretty much stopped reading at the point  
because I knew that having to use subprocesses would be a show stopper  
for me.

> TAP is a very perl centric protocol in my experience. From your
> mentioning TAP I assumed you would be grabbing one of the mature TAP
> frameworks that do test reporting, which is what prompted my  
> comment :).

Ah, I only mentioned it because the name was being slung around at  
pycon as something to look into. Better to reuse than reinvent. ;)

> Further to that, I mean that the python xUnit model for describing  
> tests
> is nice to work with and keeping that model even when doing later
> analysis means that you can do immediate-run-and-analyse and also
> run,store, and analyse simply by changing what 'suite' you plug into
> your runner.

Agreed. I had actually been considering using the standard library  
logger because it can support multiple output sinks and has support  
for a number of different transports (files, sockets, HTTP, etc.).

> A little code may help make this clear - here is 'run an report' in
> typical unittest:
>
> suite = make_suite()
> result = TextTestResult(sys.stdout)
> suite.run(result)
>
> here is 'run and log to a file' with subunit
> suite = make_suite()
> stream = file('tests.log', 'wb')
> result = TestProtocolClient(stream)
> suite.run(result)
> stream.close()
>
> here is 'process a log and report on it' with subunit
> stream = file('tests.log', 'rb')
> suite = ProtocolTestCase(stream)
> result = TextTestResult(sys.stdout)
> suite.run(result)
> stream.close()

I'm a little bit confused about the differences between the 2nd and  
third examples, perhaps because I don't get ProtocolTestCase taking a  
file parameter and TextTestResult also taking a file. Some part of the  
plumbing here is confusing me.

--Doug