[TIP] pytest-2.2.0: test marking++, parametrization++ and duration profiling
holger at merlinux.eu
Fri Nov 18 11:10:07 PST 2011
py.test 2.2.0: test marking++, parametrization++ and duration profiling
pytest-2.2.0 is a test-suite compatible release of the popular
py.test testing tool. Plugins might need upgrades. It comes
with these improvements:
* easier and more powerful parametrization of tests:
- new @pytest.mark.parametrize decorator to run tests with different arguments
- new metafunc.parametrize() API for parametrizing arguments independently
- see examples at http://pytest.org/latest/example/parametrize.html
- NOTE that parametrize() related APIs are still a bit experimental
and might change in future releases.
* improved handling of test markers and refined marking mechanism:
- "-m markexpr" option for selecting tests according to their mark
- a new "markers" ini-variable for registering test markers for your project
- the new "--strict" bails out with an error if using unregistered markers.
- see examples at http://pytest.org/latest/example/markers.html
* duration profiling: new "--duration=N" option showing the N slowest test
execution or setup/teardown calls. This is most useful if you want to
find out where your slowest test code is.
* also 2.2.0 performs more eager calling of teardown/finalizers functions
resulting in better and more accurate reporting when they fail
Besides there is the usual set of bug fixes along with a cleanup of
pytest's own test suite allowing it to run on a wider range of environments.
For general information, see extensive docs with examples here:
If you want to install or upgrade pytest you might just type::
pip install -U pytest # or
easy_install -U pytest
Thanks to Ronny Pfannschmidt, David Burns, Jeff Donner, Daniel Nouri,
Alfredo Doza and all who gave feedback or sent bug reports.
notes on incompatibility
While test suites should work unchanged you might need to upgrade plugins:
* You need a new version of the pytest-xdist plugin (1.7) for distributing
* Other plugins might need an upgrade if they implement
the ``pytest_runtest_logreport`` hook which now is called unconditionally
for the setup/teardown fixture phases of a test. You may choose to
ignore setup/teardown failures by inserting "if rep.when != 'call': return"
or something similar. Note that most code probably "just" works because
the hook was already called for failing setup/teardown phases of a test
so a plugin should have been ready to grok such reports already.
Changes between 2.1.3 and 2.2.0
- fix issue90: introduce eager tearing down of test items so that
teardown function are called earlier.
- add an all-powerful metafunc.parametrize function which allows to
parametrize test function arguments in multiple steps and therefore
from indepdenent plugins and palces.
- add a @pytest.mark.parametrize helper which allows to easily
call a test function with different argument values
- Add examples to the "parametrize" example page, including a quick port
of Test scenarios and the new parametrize function and decorator.
- introduce registration for "pytest.mark.*" helpers via ini-files
or through plugin hooks. Also introduce a "--strict" option which
will treat unregistered markers as errors
allowing to avoid typos and maintain a well described set of markers
for your test suite. See exaples at http://pytest.org/latest/mark.html
and its links.
- issue50: introduce "-m marker" option to select tests based on markers
(this is a stricter and more predictable version of '-k' in that "-m"
only matches complete markers and has more obvious rules for and/or
- new feature to help optimizing the speed of your tests:
--durations=N option for displaying N slowest test calls
and setup/teardown methods.
- fix issue87: --pastebin now works with python3
- fix issue89: --pdb with unexpected exceptions in doctest work more sensibly
- fix and cleanup pytest's own test suite to not leak FDs
- fix issue83: link to generated funcarg list
- fix issue74: pyarg module names are now checked against imp.find_module false positives
- fix compatibility with twisted/trial-11.1.0 use cases
More information about the testing-in-python