[TIP] Tipp: Ordering Test Files

[Ned Batchelder]

Can you say more about why you want the tests to run in a particular 
order?  Often once the test suite grows large, pytest-xdist enters the 
picture, and there isn't a single stream of tests to order anyway.

--Ned.


On 10/9/18 11:57 AM, Daniel Knüttel wrote:
> Hi testers,
>
> this is something I came across several times now and it really helped
> me and my colleagues.
>
> Pytest executes the tests is alphanumeric order. This can be used to
> order the tests in a meaningful order (often it is reasonable to
> execute the tests that way).
>
> I suggest the following naming scheme (adapted from the POSIX
> configuration file naming scheme):
>
> 	test_<XXX>_<name>.py
>
> where ``<XXX>`` is an integer value ranging from ``000`` to ``999`` and
> ``<name>`` is the name you would usually give your test. I usually
> associate the second digit with a module.
>
> Examples:
>
> 	test_000_io_block_buffer.py
> 	test_001_io_block_read.py
> 	test_002_io_block_write.py
> 	test_010_bootsector.py
>
> Here is a real life example:
>
> 	============================= test session starts ==============================
> 	platform linux -- Python 3.5.3, pytest-3.4.0, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python3
> 	cachedir: .pytest_cache
> 	rootdir: /home/daniel/informatik/c/bytecode_interpreter/assembler, inifile:
> 	collected 12 items
> 	
> 	test/test_010_filecontext.py::test_getc_ungetc PASSED                    [  8%]
> 	test/test_011_tokenize.py::test_tokenize_1 PASSED                        [ 16%]
> 	test/test_011_tokenize.py::test_tokenize_2 PASSED                        [ 25%]
> 	test/test_011_tokenize.py::test_tokenize_3 PASSED                        [ 33%]
> 	test/test_011_tokenize.py::test_tokenize_4 PASSED                        [ 41%]
> 	test/test_012_opcodes.py::test_make_opcodes PASSED                       [ 50%]
> 	test/test_013_util.py::test_can_be_mark PASSED                           [ 58%]
> 	test/test_013_util.py::test_can_convert_to_int PASSED                    [ 66%]
> 	test/test_013_util.py::test_autoint PASSED                               [ 75%]
> 	test/test_020_basic_parsing.py::test_commands PASSED                     [ 83%]
> 	test/test_020_basic_parsing.py::test_mark PASSED                         [ 91%]
> 	test/test_020_basic_parsing.py::test_set_directive PASSED                [100%]
> 	
> 	========================== 12 passed in 0.02 seconds ===========================
>
> Let me know your thoughts on this naming scheme.
>