[TIP] Declarative style acceptance tests (was: Using doctest for functional tests / user stories)

Michael Foord fuzzyman at voidspace.org.uk
Mon Jan 6 15:19:40 PST 2014

On 6 Jan 2014, at 22:14, Paul Moore <p.f.moore at gmail.com> wrote:

> On 6 January 2014 21:50, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>>>> Or is the whole idea of structuring the user stories as testable
>>>> documents considered bad practice? I'm finding it very difficult to
>>>> formulate good functional tests in the unittest style, but I don't
>>>> want to switch to doctest if it's going to cause me problems as my
>>>> development gets more complex.
>>> You're right, unit tests are not suited for writing acceptance tests.
>> Actually I've found unittest a perfectly good framework for writing acceptance tests. Several times.
> The thing I've found frustrating when writing acceptance tests (see!
> I'm trying to use the right words! :-)) is that unittest test cases
> are isolated from each other. That's a positive feature when I'm
> writing unit tests, but when I write an acceptance test, I feel like I
> want a certain level of interdependence. Probably I'm wrong in
> thinking that, but it tends to make it hard for me to get my head
> round writing tests with the proper granularity.
> For example:
> class CreationTest(TestCase):
>    def test_schema_create(self):
>        "Creating the DB schema initialises the application tables"
>        # Using a temporary database here
>        db = myapp.DB()
>        # I create the schema
>        db.create_schema()
>        # There should be some tables in there now...
>        self.assertEqual(len(db.tables()), 4)
> class AddTest(TestCase):
>    def setUp(self):
>        # Set up the DB for the tests - this essentially duplicates
> test_schema_creation above
>        self.db = myapp.DB()
>        self.db.create_schema()
>    def test_add_package(self):
>        self.db.add_package("foo", "1.0")
>        self.assertEqual(self.db.query_package("foo"), "1.0")
> That duplication in setting up the DB feels wrong to me. But I can't
> think of any way to refactor it. A "narrative" style of testing (which
> goes step by step through the process) would fail in the same places,
> but without the duplication.
> Of course, I could write the narrative as a single test case:
> class EnteringDataScenario1(TestCase):
>    def test_create_db_and_enter_some_data(self):
>        db = myapp.DB()
>        db.create_schema()
>        self.assertEqual(len(db.tables()), 4)
>        db.add_package("foo", "1.0")
>        assertEqual(self.db.query_package("foo"), "1.0")
> But that starts to feel like the test is too long to be a "proper"
> unittest method, particularly as the user story becomes more
> elaborate.
> Am I worrying too much about repetitive code in my tests?

I think code duplication matters a lot less in tests than it does in production code. You tend to refactor tests less (note I didn't say that you never refactor tests...) and you want each test to be fairly "self explanatory". You can still use abstraction to reduce duplication and increase readability of course.

For example, if you want a more narrative style of test:

class Base(object):
    def create_db(self):
       # Using a temporary database here
       db = myapp.DB()
    def assert_tables_exist(self):
       self.assertEqual(len(db.tables()), 4)       

class CreationTest(Base, TestCase):
   def test_schema_create(self):
       "Creating the DB schema initialises the application tables"
       # I create the schema
       # There should be some tables in there now...

Your test now has a very similar narrative style to a dedicated acceptance test framework - but you're only using abstractions rather than going through layers of parser -> test definitions -> custom runner (etc). 


> Paul


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 

More information about the testing-in-python mailing list