[TIP] Sclara is a Python testing DSL

Staple, Danny (BSKYB) DStaple at nds.com
Mon Mar 26 06:40:01 PDT 2012


Hmm - that should have been
with it("should throw bar"):

And the semicolon crept in because I've been switching languages a fair bit today.

But I see your point - for the most part, I have been using unittest. I do like the hierarchy though as it represents a hierarchy of behaviour and state. Consider an object representing a hardware resource. It can be constructed, then locked/unlocked (allowing other users of it), and then other state, when locked can be set.

I've then got first the basic device, the construction of which is one behaviour I've described. Then I have a constructed device, and all other tests apply to that. In those, I've the Locked behaviour, and the unlocked behaviour. The unlocked behaviour has the test for locking a device, and tests to ensure operations like unlocked, setting its state throw exceptions, while querying the state is permitted. The locked behaviour should permit  setting state, still permit querying state, and permit unlocking - it should not allow the device to be locked again.  These hierarchical behaviours share semantics, and also share setup/teardown (before/after) behaviour at different levels. Ie - setup for any mocks needed to construct the thing, setup a constructed object, and setup a locked object. Nesting could give a more expressive means of combination, thus meaning less duplication. I realise this could be done with helpers of other kinds - but I've seen in both a very brief and very clear test written this way.

Danny

From: Ned Batchelder [mailto:ned at nedbatchelder.com]
Sent: 26 March 2012 13:48
To: Staple, Danny (BSKYB)
Cc: 'Alfredo Deza'; Chris.Wesseling at cwi.nl; John MacKenzie; testing-in-python at lists.idyll.org
Subject: Re: [TIP] Sclara is a Python testing DSL

On 3/26/2012 8:28 AM, Staple, Danny (BSKYB) wrote:
Hmm - I am thinking that a python dsl similar to JS's Jasmine with matchers would be doable.

with describe("foo"):
  It("should throw bar"):
    expect( lambda : foo.doStuff()).toRaise(BarException);
I've never seen the value in ladling English all over my test cases.  How is your code better than what you can currently do with unittest?:

    def test_should_throw_bar(self):
        """foo should throw bar"""
        with self.assertRaises(BarException):
            foo.doStuff()

Not to mention, the snippet you presented isn't valid Python, yet involves a lambda...

--Ned.



There definitely are many python test things other than doctest and unittest (Freshen, Lettuce, PySpec, spec plugin for nose, should-dsl for some). The trick to acceptance is compatibility probably, make sure that there is a nose plugin, that it works with Micheal Foord's mock, that doesn't do such strange magic that it taints the global namespace and makes stuff behave incorrectly.

From: testing-in-python-bounces at lists.idyll.org<mailto:testing-in-python-bounces at lists.idyll.org> [mailto:testing-in-python-bounces at lists.idyll.org] On Behalf Of Alfredo Deza
Sent: 26 March 2012 12:08
To: Chris.Wesseling at cwi.nl<mailto:Chris.Wesseling at cwi.nl>
Cc: John MacKenzie; testing-in-python at lists.idyll.org<mailto:testing-in-python at lists.idyll.org>
Subject: Re: [TIP] Sclara is a Python testing DSL


On Mon, Mar 26, 2012 at 3:56 AM, <Chris.Wesseling at cwi.nl<mailto:Chris.Wesseling at cwi.nl>> wrote:
On 2012-03-24T16:00:06-0500, John MacKenzie wrote:
> http://github.com/198d/sclara
[...]
> There are failures in both as a demonstration of how this interacts with a
> TestRunner. The first one proves mainly that this is just Python; if you
> use one of the provided runners you can execute your test files with the
> Python interpreter (a big goal after looking at a tool like komira for this
> sort of thing). The latter demonstrates that a nose plugin is possible and
> probably implies that a py.test plugin is doable to.
Could you point me to "komira"? Your e-mail turns up as the only
relevant hit using my search-fu.

Konira is a Testing DSL framework [0] that I started working on about a year and a half ago. It
is not valid Python (hence the DSL connotation) but it is translated to it when tests are run
via de included Test Runner or with the py.test [1] plugin for it.

The problem that I see these solutions (including Konira) bring to the table is that they all try to
create a flexible, descriptive and terse domain for writing tests, and while some of them have a
significant gain over current testing standards, they fail when the complexity goes beyond
asserting something that has a nice description.

It might be heretic to raise this in a testing list for Python, but I also believe that the Python community
is not as pro-innovation in the testing environment as other communities are, which is detrimental to the fact
that there are a few people who don't like the standard testing framework and are looking forwards to an
alternative.




> Beyond sclara, I'm considering the idea that, more than a testing
> framework, I stumbled on an interesting pattern for building generic DSLs
> in Python (more on this soon, hopefully).
>
> Thoughts?
I started reading the tests, to see if the DSL provides the clarity we need in
this domain:

examples/test_sclara.py line 14:

   with test('does not have access to inner setup context') as context:
       try:
           context.bar
       except AttributeError:
           pass


Will this fail if no AttributeError is thrown?

And line 45:

   with test('has access to inner setup context') as context:
       assert context.foo == 'bar'

   with test('has access to outer setup context') as context:
       assert context.bar == 'foo'

Shouldn't "inner" and "outer" be swapped here?
The outer setup adds foo and baz attributes.
The inner setup adds the bar attribute to the context and changes the baz value.


What I see as a pitfall for users with this idiom, is that it might invite to
breaking isolation.

Interesting, though.

[0] http://konira.cafepais.com/docs/index.html
[1] http://pypi.python.org/pypi/pytest-konira

--
Chris Wesseling
Centrum Wiskunde & Informatica (CWI)
https://www.cwi.nl/people/ccw

_______________________________________________
testing-in-python mailing list
testing-in-python at lists.idyll.org<mailto:testing-in-python at lists.idyll.org>
http://lists.idyll.org/listinfo/testing-in-python


________________________________

**************************************************************************************
This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmaster at nds.com<mailto:postmaster at nds.com> and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes. To protect the environment please do not print this e-mail unless necessary.

NDS Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United Kingdom. A company registered in England and Wales. Registered no. 3080780. VAT no. GB 603 8808 40-00
**************************************************************************************




_______________________________________________

testing-in-python mailing list

testing-in-python at lists.idyll.org<mailto:testing-in-python at lists.idyll.org>

http://lists.idyll.org/listinfo/testing-in-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20120326/eec3ea5c/attachment-0001.htm>


More information about the testing-in-python mailing list