[TIP] running unittests on multiple OS, including non-Posix
John Yeuk Hon Wong
gokoproject at gmail.com
Fri Feb 28 13:43:34 PST 2014
On 2/28/14 3:56 PM, Albert-Jan Roskam wrote:
> I would like to run my unittests (--> nose) on multiple platforms. I
> also would like to test different python versions and implementations
> on each platform (--> tox ). These platforms also include Windows,
> so Travis CI or Docker is not an option, AFAIK. I was thinking about
> using Vagrant  to fire up VirtualBox  VMs for each
> platform-to-be-tested, then either (a) fire up nose or tox through SSH
> or (b) (the easy way) prepare each VM such that nose/tox is fired up
> right when the OS starts (ie., edit .bashrc, autoexit.bat or whatever
> it is called for the OS at hand). But this all feels like reinventing
> the wheel. Can you recommend a package or strategy to use? Beautiful
> is better than ugly, although practicality beats purity. In other
> words: it would nice if it is The Right Way, but I am a lazy bastard
> so I am also open for quick-and-dirty approaches ;-)
I don't think there is a standalone, open standard how to do this kind
Testing is very project dependent and I often run into problems testing
on external servers, hence why I prefer not to build on Travis, or drone.io.
Docker is a container and as far as I (and you) know, Docker does not
run on Windows (and Docker is based on LXC technology).
A lot of people run Jenkins to manage continuous integration. Basically,
Jenkins is a job management software. I don't really run Jenkins these
days. But you can check this out:
Basically you can boot slaves on demand when a test is triggered.
For larger software, I have seen people run openstack to handle creating
new vms and putting them into farms.
Bottom line is:
For Linux, create a docker container of everything you need to run your
job (call this a base docker image). Export the docker. Each time you
run a test, tell Jenkins / your own custom bash script to create a new
docker instance based on the base image. Run the job and report the
result then remove the instance if you want to. I mention to use docker
here because it's cheaper to launch docker instance. But for
consistency, you may consider going for full vagrant vm as windows one
would use vagrant vm. It's up to you.
For windows, just create a windows machine, bootstrap the machine,
export the machine into a vagrant box/ova file. On each test, boot up
the new VM and then run the tests.
That's how I handle things right now, but if anyone have any exceptional
method, do share please.
In my opinion, the best testing tool is the one that works for your
need. Generic tool often makes things more complicated.
To add complexity, some people go as far as using puppet/chef/ansible
instead of vanilla bash/shell scripts. That's great if you have a big
test farms. If I were you, first get multiple platform testing working,
then start transiting. I don't know how many hours I lost trying to
invent complication :)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the testing-in-python