<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=us-ascii" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
On 27/02/2010 20:55, Robert Collins wrote:
<blockquote cite="mid:1267304156.25053.23.camel@lifeless-64" type="cite">
<pre wrap="">On Sat, 2010-02-27 at 12:13 -0800, C. Titus Brown wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi all,
I've updated the README for my SomePackage example code,
</pre>
</blockquote>
<pre wrap="">
So, my key points are, I guess:
python -m somepackage.tests.run - isn't very useful:
- doesn't define getting JUnitXML (is it a command line option? should
you use subunit? should you use nose-junit, should you use
xmltestrunner?)
</pre>
</blockquote>
<br>
It is no less useful than "setup.py test" and Titus intends it to be
for a different situation - post installation where there is no
setup.py. It provides a standard entry point for running tests. The
issue of command line options is real but orthoganal.<br>
<br>
<br>
<blockquote cite="mid:1267304156.25053.23.camel@lifeless-64" type="cite">
<pre wrap="">
- doesn't define command line parameters except implicitly as 'the
unittest ones' based on your sample code; in which case aiming for API
conformance is more useful.
These concerns also apply to setup.py test. As you say '
- Titus Brown: I also don't really understand what the arguments to
'setup.py test' are.'
For integration with CI tools, the primary, defacto method is 'get a
JUnit XML file on disk', with everything else secondary.
</pre>
</blockquote>
<br>
So, how do we provide command line options to test running with our
standard-mechanisms is a big question. For continuous integration
(pony-build) standard output (unittest or nose run with verbose output)
could be parsed and converted into XML. Not ideal but at least
demonstrates that it *could* be done without *having* to standardise
command line handling.<br>
<br>
<blockquote cite="mid:1267304156.25053.23.camel@lifeless-64" type="cite">
<pre wrap="">
API conformance: The unittest API itself is extraordinarily flexible for
the sorts of things we're talking about here; I can present a unittest
TestSuite which will start up instances on EC2 and execute the tests
there: its very powerful and very lean: define countTestCases(*), run,
debug(arguably option), and thats nearly all of it: you can, though its
not desirable, ignore the test result passed into run, and not report
through the unittest API: your tests will still /run/. Michael Foord,
others and I all want to make this even better: to make things like
'TestRunner' go away (it can now with startTestRun and stopTestRun,
there is no need for it at all, just hasn't been done in core).
</pre>
</blockquote>
<br>
Whilst I agree with you (except debug is an abomination that should go
away) I don't see how it is relevant here. Perhaps just a worthwhile
observation supporting the load_tests protocol?<br>
<br>
<blockquote cite="mid:1267304156.25053.23.camel@lifeless-64" type="cite">
<pre wrap="">
I disagree with Michael on test_suite being unsupported/whatever: lots
of people use it. Its nowhere near as convenient as load_tests: its
kindof a 1st gen attempt at that. Nevertheless, test_suite gets used -
and supported.
</pre>
</blockquote>
<br>
Well, it is a legacy protocol. If it goes into distutils2 then it will
never go away. Test suites that want to be compatible with both can
simply do:<br>
<br>
def load_tests(loader, tests, pattern):<br>
return test_suite()<br>
<br>
So I still see no reason to support it in distutils2. Not entirely my
decision of course.<br>
<br>
<br>
<blockquote cite="mid:1267304156.25053.23.camel@lifeless-64" type="cite">
<pre wrap="">
Oh, I should note that I don't think setup.py knowing how to do all the
funky things developers may do is needed: if the primary goals are:
- to make it easy for distributors and casual tweak-local-for-me users
to be sure the thing builds ok
- to make it more routine to incorporate into CI tools like Hudson
then being able to make it do what 'bzr selftest --no-plugins
--parallel=subprocess -s bt.per_repository -x CommitBuilder --subunit'
does really isn't important. Being able to (in the first case) make it
do 'bzr selftest' and (in the second case) 'bzr selftest --subunit |
subunit2junitxml -o tests.xml' are the important features.
I'm still not clear on the step from 'setup.py test' to 'I can has XML'.
</pre>
</blockquote>
<br>
Agreed, not an easy question and still one that is open. Particularly
relevant for Titus I guess.<br>
<br>
Michael<br>
<br>
<blockquote cite="mid:1267304156.25053.23.camel@lifeless-64" type="cite">
<pre wrap="">
Hope this helps, I'm happy to put code up if that would help more.
-Rob
</pre>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
testing-in-python mailing list
<a class="moz-txt-link-abbreviated" href="mailto:testing-in-python@lists.idyll.org">testing-in-python@lists.idyll.org</a>
<a class="moz-txt-link-freetext" href="http://lists.idyll.org/listinfo/testing-in-python">http://lists.idyll.org/listinfo/testing-in-python</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.ironpythoninaction.com/">http://www.ironpythoninaction.com/</a>
<a class="moz-txt-link-freetext" href="http://www.voidspace.org.uk/blog">http://www.voidspace.org.uk/blog</a>
READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
</pre>
</body>
</html>