[TIP] Tasting the Cheeseshop

Barry Warsaw barry at python.org
Mon Oct 17 14:30:19 PDT 2011


On Oct 15, 2011, at 05:03 AM, holger krekel wrote:

>thanks for tackling this and sharing your thoughts!

Thank *you* for some awesome software!

>- tox could easily grow XML or other standardized output but it would
>  not report results per test function/item - this is left to
>  the individual test tool that is invoked.  If that is standardized
>  (ad we could recard junitxml as such for now) then we could of course
>  embed the junitxml output into the prospective tox output if that helps.

That seems to be the best direction to head in.  Let tox report what it can,
but I agree that the individual test reports should be left up to the
package's selected test runner.  It sounds like there's at least general
agreement that junitxml or subunit is the format to support.

>- i don't think it's a good idea to incorporate test standards 
>  into setup.py because 
>  a) setup.py is executable code, is not introspectable and

Right, but I'm using setup.py as a backward compatible inferior stand-in for
pysetup and setup.cfg.  Eventually we'll all be using packaging, so let's
design that for maximum happiness.  Then I think we can hack in support for
older Pythons and legacy packaging tools.  E.g. could we get the test related
arguments to setup() into egg info?

>  b) setup.py is meant to go away with the new distutils2 for a pure
>  declarative approach (tox's config could easily merge in there) and 

+1 for setup.cfg.

>  c) it's kind of requiring a specific unittest-pkg based model where tests
>  are inlined into the package. There are reasons why substantial amount
>  of projects use or need different approaches.

Right.  We'll never get 100% coverage, but I also think the majority of
packages on PyPI could squeeze into a standard we promote.  What I'd like to
avoid is *unnecessary* ad-hoc guesswork to figure out how to run a package's
test suite for 80% of the packages out there.  That's what I think we have
today, and a mass of heuristics is really not conducive to systematic PyPI
testing.  It certainly won't help with some of the other use cases I'm
contemplating.

>FWIW i am happy to share moving tox forward with other authors - Michael
>Foord has expressed interest there as well in the past and Kumar (nose
>maintainer) gave a talk about it at last PyCon.  I'd be glad if tox
>evolved to introduce a test running standard finally merging with setup.cfg.

Cool.  We have plenty of time to plan for Pycon 2012 sprints. :)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20111017/4fdfeaaf/attachment.pgp>


More information about the testing-in-python mailing list