[TIP] using unittest for other tasks

[John Wong]

> More specifically, unittest is great for unit testing: isolating a small
unit of code and testing whether it gives expected outputs for given
inputs.

Did you mean unittest as in unittest module? It was probably a Python port
of the Java's Junit, IRRC.

I don't think there is anything to prevent someone to use the module to do
functional testing. Unittest module is really just a set of commands such
as assertEqual and what not. It is so bare metal that i don't think it was
designed to be just unit testing.

> I manage several hundred systems and I am currently using Nagios but
instead I was wondering if unittest would be OK for something like this.

For system behavior testing, you need to be very specific what you want to
test. Have you ever heard of Puppet, Chef, Ansible or Salt-stack? They are
known as configuration management tool. The reason people use configuration
management system is because they want to ensure systems are continuously
configured at the state you want they to be in. Take Ansible for example, I
can do this:

- hosts: proxy
  sudo: True
  tasks:
      - name: Ensure Nginx is installed
        apt: pkg=nginx state=present
      - name: Configure nginx.conf
        template: src=templates/nginx.proxy.conf.j2
dest=/etc/nginx/nginx.conf

and in nginx.proxy.conf.j2 you might have a template like this
worker count = {{ number_of_cpu }}

Don't mind the correct syntax here, just a high level.

What does this do for you?
1. if you just boot up a new vm, and running the tasks above, nginx will be
installed and the tool will configure nginx.conf and use whatever # of cpu
found for that vm to be the worker count. If 20 servers are 4 core you get
4, and if the rest are 2 cores, you get 2.
2. If you alreayd have run the script before, you can run again, and unless
you upgrade your system to have more cores, running the same script on the
same machine twice should yield no change, ie. idempotent.

As you can see from #2, this is great for continuous deployment and server
management. I can run the same configuration setup on the same set of
machine as many times as i want and should yield no side effect, PROVIDED
that the system hardware hasnt changed, and that the variables haven't
changed.

Say you have a task that fetch a github repo changes, you can write a task
to fetch either the latest or a specific changes. This ensures system is
behaving the way you want. In another words, YOU write a bunch of scripts
to tell the system how to setup and beahve, and ask the tool to run the
scirpts againsts some servers and the tool should not do different thing as
long as the tasks haven't changed and system dependent variables stay the
same.

Get it?

Write tests to test system is not rare, but there must be a very good
reason. The only one I have seen is someone trying to debug system problem,
that's all. Others write system tools such as new nagios modules. To test
nagios is installed, or a particular configuration is used, writing

def test_nagios_fileA_used(self):
    ip?
    sudo privilege?

See, that's tough, huh. And how do you assert? Oh 'this line should be
present' in f.read()

You can essentially running a configuration tool.

John




On Wed, Jul 24, 2013 at 5:37 PM, Ben Finney <ben+python at benfinney.id.au>wrote:

> Rita <rmorgan466 at gmail.com> writes:
>
> > I know that unittest is great for testing code.
>
> More specifically, unittest is great for unit testing: isolating a small
> unit of code and testing whether it gives expected outputs for given
> inputs.
>
> > I was wondering if anyone is using it to test system consistency. I
> > manage several hundred systems and I am currently using Nagios
>
> I think Nagios is more for monitoring, not automated testing. Is that
> not correct?
>
> > but instead I was wondering if unittest would be OK for something like
> > this.
>
> For testing system behaviour, the unittest module is a poor fit since it
> is designed to test isolated units of code at the function-call level.
>
> You would do better to look at behavioural testing tools, like Behave
> <URL:https://pypi.python.org/pypi/behave> that can work from assertions
> about the behaviour of the whole system.
>
> --
>  \       “We must find our way to a time when faith, without evidence, |
>   `\    disgraces anyone who would claim it.” —Sam Harris, _The End of |
> _o__)                                                     Faith_, 2004 |
> Ben Finney
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20130724/780805b8/attachment.htm>