[TIP] using unittest for other tasks

[Eric Larson]

Ben Finney writes:

> Rita <rmorgan466 at gmail.com> writes:
>
>> I know that unittest is great for testing code.
>
> More specifically, unittest is great for unit testing: isolating a small
> unit of code and testing whether it gives expected outputs for given
> inputs.
>
>> I was wondering if anyone is using it to test system consistency. I
>> manage several hundred systems and I am currently using Nagios
>
> I think Nagios is more for monitoring, not automated testing. Is that
> not correct?
>
>> but instead I was wondering if unittest would be OK for something like
>> this.
>
> For testing system behaviour, the unittest module is a poor fit since it
> is designed to test isolated units of code at the function-call level.
>
> You would do better to look at behavioural testing tools, like Behave
> <URL:https://pypi.python.org/pypi/behave> that can work from assertions
> about the behaviour of the whole system.

I doubt I'd use unittest for this sort of thing. Behave also seems likes
it might be a bit too specific, even though conceptually it is probably
a good fit.

I would consider pytest just b/c it has a flexible test discovery
mechanism. Pytest fixtures seems like they would be really helpful as
well doing the setup for talking to sets of hosts.

Obviously, it wouldn't be a traditional test suite, but if you did have
a jenkins task running every so often that ran the test suite, you could
get a nice UI with specific errors you could hopefully track down.

I do the same sort of thing for other tasks I'm iterating on. It is easy
to run pytest in my editor (Emacs + pytest.el), so often times I'll find
it is easy to write a test that I run and re-run until I get what I
need. A good example would be screen scraping some HTML and pulling out
some data. The setup and teardown make good hookpoints to cache the
results and I can build on functionality using the pytest fixtures.

I'm sure this is doable in nose and unittest as well. I specifically
mention pytest because I do think it has some features that make it
slightly more helpful doing what I *think* you're doing. 

Best of luck!

Eric