[TIP] Sclara is a Python testing DSL

[John MacKenzie]

Thanks, guys. I'll try and respond the best I can to everything I've read:

examples/test_sclara.py line 14:
>
>    with test('does not have access to inner setup context') as context:
>        try:
>            context.bar
>        except AttributeError:
>            pass
>
>
> Will this fail if no AttributeError is thrown?


Good point. That should probably raise an AssertionError if no
AttributeError is thrown to show that it worked when it shouldn't have.

And line 45:
>
>    with test('has access to inner setup context') as context:
>        assert context.foo == 'bar'
>
>    with test('has access to outer setup context') as context:
>        assert context.bar == 'foo'
>
> Shouldn't "inner" and "outer" be swapped here?
> The outer setup adds foo and baz attributes.
> The inner setup adds the bar attribute to the context and changes the baz
> value.


I'll make sure to fix that. Thanks.


It might be heretic to raise this in a testing list for Python, but I also
> believe that the Python community
> is not as pro-innovation in the testing environment as other communities
> are, which is detrimental to the fact
> that there are a few people who don't like the standard testing framework
> and are looking forwards to an
> alternative.
>

I get this feeling, too; not sure how to remedy this (assuming it's an
actual problem).



> Hmm – I am thinking that a python dsl similar to JS’s Jasmine with
> matchers would be doable.****
>
> ** **
>
> with describe(“foo”):****
>
>   It(“should throw bar”):****
>
>     expect( lambda : foo.doStuff()).toRaise(BarException);
>

The more I thought about this the more I liked it. You could probably push
an API like this as a tool to be used regardless of what testing framework
you're using. If it was pluggable, I think you might have something pretty
powerful.


I've never seen the value in ladling English all over my test cases.  How
> is your code better than what you can currently do with unittest?:
>
>     def test_should_throw_bar(self):
>         """foo should throw bar"""
>         with self.assertRaises(BarException):
>             foo.doStuff()


Personally, I just hate writing method names while testing. I hate it. It
makes me think like a programmer at exactly the wrong time. At that point
in time, I want to be thinking abstractly about the system under test and
what it is that it is supposed to do. When I'm using classes and methods to
do that, I'm doing OOP and it just pains me. Now, I'm not saying OOP is
something never to do, but I don't want to do it there.


Could that complexity be a code smell?
> Given enough context every assertion could have a nice description. Or
> couldn't they?


I think that's kind of the goal. I'm pretty sure it's generally agreed upon
that 1 assertion per test is generally what people mean when talking about
unit tests and to have an english statement that is either true or false
based on the outcome of that assertion is pretty neat (I think).


I’ve then got first the basic device, the construction of which is one
> behaviour I’ve described. Then I have a constructed device, and all other
> tests apply to that. In those, I’ve the Locked behaviour, and the unlocked
> behaviour. The unlocked behaviour has the test for locking a device, and
> tests to ensure operations like unlocked, setting its state throw
> exceptions, while querying the state is permitted. The locked behaviour
> should permit  setting state, still permit querying state, and permit
> unlocking – it should not allow the device to be locked again.  These
> hierarchical behaviours share semantics, and also share setup/teardown
> (before/after) behaviour at different levels. Ie – setup for any mocks
> needed to construct the thing, setup a constructed object, and setup a
> locked object. Nesting could give a more expressive means of combination,
> thus meaning less duplication. I realise this could be done with helpers of
> other kinds – but I’ve seen in both a very brief and very clear test
> written this way.


Yes. This is exactly the idea.


Interesting we had a discussion about a theoretical tool similar to this a
> while back.


Hah. A lot of that is identical to what I had done at various stages during
development; kind wild.


How is this similar to or different than Lettuce?


I believe Lettuce is based on Cucumber which claims to be an acceptance
testing tool. It's meant to be quite a bit higher level than something like
what I'm trying to do (which is more for unit testing) and probably best
suited for more functional/integration testing.


Again, I appreciate the feedback and discussion. I'll keep hacking and keep
you guys posted. Thanks!


On Mon, Mar 26, 2012 at 8:31 PM, Ned Batchelder <ned at nedbatchelder.com>wrote:

> On 3/24/2012 5:00 PM, John MacKenzie wrote:
>
>> http://github.com/198d/sclara
>>
>> I wrote this library based on a thought I had at the TiP BoF this year
>> (hence the name "s(anta)clara"): RSpec for Python (unoriginal idea) written
>> entirely with context managers (original idea, hopefully). I had been
>> writing Ruby for 3 years up until about December of last year and have been
>> writing Python full time since then. What I like most about RSpec is the
>> way you define and organize tests. You end up grouping similar test cases
>> in a sort of hierarchical manner (without resorting to OOP, which I hate
>> the most about writing tests with unittest) and get things like cumulative
>> setup/teardown from parent contexts while producing an english statement
>> that is either true or false after the test is run. These are the main
>> ideas I tried to capture with sclara. Context mangers, at first glance kind
>> of make sense; if you make a quick sketch of what this might look like, you
>> get something like this:
>>
>> with description('Our object under test'):
>>  with test('does something'):
>>    assert True
>>  with test('does something else'):
>>    assert True
>>  with description('when some condition is met'):
>>    with test('acts a specific way'):
>>      assert True
>>
>>
> How is this similar to or different than Lettuce?  Sorry, I have no
> experience with any of these rubyesque tools.
>
> --Ned.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20120327/538c70c4/attachment.htm>