[TIP] Django custom test runner with Jenkins

[Carl Meyer]

Hi Jonathan,

On 03/17/2012 05:21 AM, Jonathan Hartley wrote:
> I got a lot of out Carl Meyer's PyCon talk "Testing and Django":
> http://pyvideo.org/video/699/testing-and-django

Great, glad it was useful!

> One of his suggestions is a custom Django test runner to augment test
> discovery, by subclassing DjangoTestSuiteRunner, and referencing it in
> settings.TEST_RUNNER.
> 
> I went down this route a while back, but then ripped it out and reverted
> to Django's built-in test runner when we added Jenkins CI to our project
> a handful of weeks later, because Jenkins has its own test runner, that
> by itself doesn't find all the tests that my/Carl's test runner would
> discover. I lacked the wherewithall to get Jenkins to use my custom test
> runner, while still performing all the actions that Jenkins itself needs.
> 
> Am I missing a trick? Is this actually easy?

I assume you're using django-jenkins
(https://github.com/kmmbvnr/django-jenkins)? I haven't used it much, but
it looks like you could get the same effect by subclassing their
django_jenkins.tasks.django_tests.Task into your own task, and
overriding the build_suite method in much the same way you'd override
the build_suite method of DjangoTestSuiteRunner. Then you'd list your
task class in place of the default django_tests task in the
JENKINS_TASKS setting to get "manage.py jenkins" to use it.

Getting the same test discovery used for local test-running depends on
whether you're using django-jenkins' "manage.py jtest" or Django's
"manage.py test". If the former, unfortunately it's written to hardcode
the built-in "django_tests" task regardless of the contents of
JENKINS_TASKS, so you'd need to replace it with your own similar
management command that uses your task instead. If the latter, you'd
still want to set TEST_RUNNER to a DjangoTestSuiteRunner subclass that
overrides build_suite; in that case you might need to factor out the
actual discovery code into a shared function that your runner subclass
and django-jenkins task subclass could both use.

(I'd say django-jenkins ought to make this simpler; seems worth filing a
bug on it - specifically it ought to be possible to replace the
test-running task used by "manage.py jtest".)

> The alternative solution I switched to was to have every Django app's
> "tests.py" call a function which does test discovery, and then inject
> the discovered TestCase classes into the tests.py module namespace, so
> that Django's default testrunner will find tests no matter where I put
> them.

That works too; more boilerplate than I'd prefer.

> (I presume Carl's on the list: If so, thanks very much for the talk -
> I'm the guy who sat next to you at the PyCon sprints at one point and
> threatened to watch the video of your talk while sitting next to you.)

I remember :-) Thanks for the kind words about the talk.

Carl

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20120317/defd2ed0/attachment.pgp>