[TIP] alphabetical order in testing

[Brad Allen]

On Tue, Jan 19, 2010 at 2:26 PM, Alfredo Deza <alfredodeza at gmail.com> wrote:

> What would be the ideal/preferable way of dealing with methods that depend
> in the order they are being executed?

> Given that:
>
>> class TestFiles(unittest.TestCase):
>>
>>     def test_second_file(self):
>>         """I depend on z_file"""
>>
>>     def test_z_file(self):
>>         """I should run before second_file"""
>

If you really want to dictate the sequence of tests you can build a
tuple of test functions, like this:

tests = ( TestFiles.test_z_file, TestFiles.test_second_file, )

However, those functions/methods should probably be named without
'test_' so that your test runner doesn't automatically pick them up
and run them out of sequence. You'll need to redefine 'test_' with
something different like 'check_'. Your TestFiles class would also
need to be renamed, so your tests sequence would look something like
this:

tests = ( CheckFiles.test_z_file, CheckFiles.test_second_file, )

Then you can write a test generator to call each of those tests in
sequence, like this:

def test_run_sequential_tests():
    for test in tests:
        yield test


One problem with this approach is that the tests lose their name. At
least, that is what I found when we tried this at work a few months
ago using the 'nose' test runner. We created a decorator to attach a
'description' attribute which nose uses when reporting the tests.

def attach_description (func):
    def decorated_func(*args, **kwargs):
        return func(*args, **kwargs)
    decorated_func.description = func.__name__
    return decorated_func


Of course, I agree with those posting to the list who say that it's a
good idea to avoid tests with sequential dependency, because they can
be fragile and harder to maintain.  We just did it at work because we
wanted to try 'pinch point' testing, loading a large data file and
verify the database state at various points along the way in the
dataload. However, that kind of testing is time consuming and not
habit-forming. Normally our developers just run the regular unit and
integration tests with no sequential dependencies (setup and clean out
the database with each individual test).