[TIP] Declarative style acceptance tests (was: Using doctest for functional tests / user stories)

[Michael Foord]

On 6 Jan 2014, at 20:12, Ben Finney <ben+python at benfinney.id.au> wrote:

> Paul Moore <p.f.moore at gmail.com> writes:
> 
>> Is doctest considered a bad idea for this type of high-level narrative
>> style of functional test (or spec)?
> 
> Doctest is best used for testing examples that already occur in
> documentation, such as API references or tutorials.
> 
> For constructing rigorous acceptance tests, Doctest is a poor choice.
> 
>> Or is the whole idea of structuring the user stories as testable
>> documents considered bad practice? I'm finding it very difficult to
>> formulate good functional tests in the unittest style, but I don't
>> want to switch to doctest if it's going to cause me problems as my
>> development gets more complex.
> 
> You're right, unit tests are not suited for writing acceptance tests.
> 

Actually I've found unittest a perfectly good framework for writing acceptance tests. Several times.

> There are better options. It's best to have acceptance tests in as close
> a form to statements the customer can understand; this means declarative
> assertions in something close to natural English. These declarative
> assertions can then be parsed and tested by executable test code.

I'm sceptical that any real world customer anywhere, ever (almost), actually reads acceptance tests - let alone writes any.

I'm *very* sceptical that using a poorly specified, undocumented, language for writing tests, especially one that *looks* like English but is still a formal "programming language" with the strict syntax requirements that entails, is any better than just using Python.

You can make your own test framework, built on unittest, close to a "DSL" through abstracting common actions into well named methods and parameters. You can end up with a set of acceptance / functional tests that are very readable with almost one line per user action. Yet still be using "normal Python" and a normal test runner.

> 
>> If anyone has any thoughts or suggestions, or better alternatives I'd
>> be very grateful.
> 
> One that I recommend is Behave <URL:http://pypi.python.org/pypi/behave>
> which reads acceptance tests written in a declarative, formalised, but
> plain-to-read English style; and then the programmer writes test
> functions which define what the terms in those declarations mean. Behave
> then tests the teclarations using your test code, and reports the
> result.

My personal opinion (and those of others I might add), is that such frameworks (cucumber, lettuce and friends) are abominations. I realise other people disagree however...

All the best,

Michael Foord

> 
> -- 
> \       “I disapprove of what you say, but I will defend to the death |
>  `\     your right to say it.” —Evelyn Beatrice Hall, _The Friends of |
> _o__)                                                  Voltaire_, 1906 |
> Ben Finney
> 
> 
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html