<div dir="ltr"><br><div class="gmail_quote"><div dir="ltr">On Tue, Oct 9, 2018 at 10:06 AM Ned Batchelder <<a href="mailto:ned@nedbatchelder.com">ned@nedbatchelder.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Can you say more about why you want the tests to run in a particular <br>
order? Often once the test suite grows large, pytest-xdist enters the <br>
picture, and there isn't a single stream of tests to order anyway.<br>
<br>
--Ned.<br></blockquote><div><br></div><div>Indeed. I find more value in test runners that effectively randomize the order. It makes flaky tests with side effects that leak out apparent, improving the overall health of your test suite. test_b was passing only because test_a did something that made test_b's test not do what was intended? you'll find out rather than potentially hiding a bug.</div><div><br></div><div>-gps</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
On 10/9/18 11:57 AM, Daniel Knüttel wrote:<br>
> Hi testers,<br>
><br>
> this is something I came across several times now and it really helped<br>
> me and my colleagues.<br>
><br>
> Pytest executes the tests is alphanumeric order. This can be used to<br>
> order the tests in a meaningful order (often it is reasonable to<br>
> execute the tests that way).<br>
><br>
> I suggest the following naming scheme (adapted from the POSIX<br>
> configuration file naming scheme):<br>
><br>
> test_<XXX>_<name>.py<br>
><br>
> where ``<XXX>`` is an integer value ranging from ``000`` to ``999`` and<br>
> ``<name>`` is the name you would usually give your test. I usually<br>
> associate the second digit with a module.<br>
><br>
> Examples:<br>
><br>
> test_000_io_block_buffer.py<br>
> test_001_io_block_read.py<br>
> test_002_io_block_write.py<br>
> test_010_bootsector.py<br>
><br>
> Here is a real life example:<br>
><br>
> ============================= test session starts ==============================<br>
> platform linux -- Python 3.5.3, pytest-3.4.0, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python3<br>
> cachedir: .pytest_cache<br>
> rootdir: /home/daniel/informatik/c/bytecode_interpreter/assembler, inifile:<br>
> collected 12 items<br>
> <br>
> test/test_010_filecontext.py::test_getc_ungetc PASSED [ 8%]<br>
> test/test_011_tokenize.py::test_tokenize_1 PASSED [ 16%]<br>
> test/test_011_tokenize.py::test_tokenize_2 PASSED [ 25%]<br>
> test/test_011_tokenize.py::test_tokenize_3 PASSED [ 33%]<br>
> test/test_011_tokenize.py::test_tokenize_4 PASSED [ 41%]<br>
> test/test_012_opcodes.py::test_make_opcodes PASSED [ 50%]<br>
> test/test_013_util.py::test_can_be_mark PASSED [ 58%]<br>
> test/test_013_util.py::test_can_convert_to_int PASSED [ 66%]<br>
> test/test_013_util.py::test_autoint PASSED [ 75%]<br>
> test/test_020_basic_parsing.py::test_commands PASSED [ 83%]<br>
> test/test_020_basic_parsing.py::test_mark PASSED [ 91%]<br>
> test/test_020_basic_parsing.py::test_set_directive PASSED [100%]<br>
> <br>
> ========================== 12 passed in 0.02 seconds ===========================<br>
><br>
> Let me know your thoughts on this naming scheme.<br>
> <br>
<br>
<br>
_______________________________________________<br>
testing-in-python mailing list<br>
<a href="mailto:testing-in-python@lists.idyll.org" target="_blank">testing-in-python@lists.idyll.org</a><br>
<a href="http://lists.idyll.org/listinfo/testing-in-python" rel="noreferrer" target="_blank">http://lists.idyll.org/listinfo/testing-in-python</a><br>
</blockquote></div></div>