[TIP] Declarative style acceptance tests (was: Using doctest for functional tests / user stories)

[Olemis Lang]

On Mon, Jan 6, 2014 at 3:53 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 6 January 2014 20:12, Ben Finney <ben+python at benfinney.id.au> wrote:
> > Paul Moore <p.f.moore at gmail.com> writes:
> >
> >> Is doctest considered a bad idea for this type of high-level narrative
> >> style of functional test (or spec)?
> >
> > Doctest is best used for testing examples that already occur in
> > documentation, such as API references or tutorials.
>
> Thanks. I was aware that was the intended use for doctest, and I had
> also seen plenty of comments to the effect that doctest is "bad" for
> general testing. But I wasn't able to find much in the way of clear
> explanations of *why* it is bad. Obviously it depends on what sort of
> tests you're trying to do - to me, it's clearly a bad fit for unit
> tests, but I'm less clear as to why it's inappropriate for functional
> tests





> (you refer to "acceptance" tests - forgive my ignorance, I'm not
> entirely clear what the difference is here).
>
>
artifacts you'll deliver to end-users / clients not necessarily aware of
testing , programming , ... subtleties [1]_

[...]

> There are better options. It's best to have acceptance tests in as close
> > a form to statements the customer can understand; this means declarative
> > assertions in something close to natural English. These declarative
> > assertions can then be parsed and tested by executable test code.
>
> In the case of my application, the end user is basically myself. I am
> writing a command line utility. In actual fact, the CLI is essentially
> likely to be a thin wrapper around a library API, and my real focus at
> this point is on getting the API correct. Something like
>
>     db = myapp.DB()
>     db.create_schema()
>     datasource = Source("web app")
>     for batch in datasource.load(batchsize=100):
>         db.load(batch)
>
> (At the command line, that'd be something like "myapp create_schema"
> and "myapp load --batchsize=100 --source='web app'" - hence my comment
> that the CLI is a thin wrapper).
>
> Writing a "narrative" like that gives me a good idea of how I want the
> API to work, and I was asking my question because that feels so close
> to a doctest.
>

FWIW , I've been thinking since some time ago of adding in dutest support
for command line tools . Yes , this would be possible to achieve by
overriding dutest.DocTestCase methods , the runner , etc ... to achieve
that each interactive example is executed by the shell and match it's ouput
just like doctest would .

If this is close to what you are looking for (and I understood correctly)
then , yes , it's possible ; but ... there's nothing (that I know) actually
written to satisfy your requirements .

Again, there are other options out there .

[...]

-- 
Regards,

Olemis - @olemislc

Apache™ Bloodhound contributor
http://issues.apache.org/bloodhound
http://blood-hound.net

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20140106/81291785/attachment.htm>