[TIP] TiP BoF feedback & call for help

[Jorge Vargas]

Hello I'll like to thank everyone for their feedback. In short I did a
couple of things suggested here.

Started with the profiling and nailed it down to be a database issue,
After looking into it a little more the most time was spend narrowing
down the problem to our fixtures. They where all disorganized and we
loaded too much stuff...
So we rearranged that and it got us a 50% improvement. Then some more
tweaks here and there and I was able to run the suite in under 6
minutes on my mac.
However our jenkins server was still slow. And after investigating
some more there I came up to the conclusion that the problem was the
hardware (too slow virtualized CPU). So we queued up the tasks for
this project to execute the db tests (we test against two engines)
lineally rather than in parallel. The end result is that even though
we are running one job first and the other one after the overall time
is less. On top of that we get the added feedback of getting a report
of failures earlier.

So right now our feedback is down to 7-9 minutes for the first
database and about the same time for the second one. With local runs
of less than 6 sometimes up to 5 minutes.

Cheers and keep on testing :)

On Tue, Mar 15, 2011 at 5:15 PM, Gary Bernhardt
<gary.bernhardt at gmail.com> wrote:
> On Mon, Mar 14, 2011 at 12:11 PM, Jorge Vargas <jorge.vargas at gmail.com> wrote:
>> So who is wrong? me? us? jenkins? django? unittest ? Who to blame ?
>> I'll love to run my tests more but 30 minutes feedback is just crazy.
>
> I pretty much agree with everything that's been said. I'd summarize as:
>
> - Django's encouragement of tight coupling can lead to slow tests.
> - But, Django isn't *making* you write slow tests; it's just making
> them easy. It's possible to write fast tests in Django.
> - The test runner probably isn't related to your problem.
> - Parallelizing can treat the symptom temporarily, but will allow the
> core problem to continue getting worse.
> - Writing more focused tests with mocks will solve your current problem.
> - But, if you use too many mocks too fast, or use them without
> thinking carefully, it'll bite you down the road.
> - And, if you write huge classes/functions and test them with mocks,
> the combination of bad design and isolated tests will make changing
> your system hard.
>
> Here are some things to think about: How can you pull the logic in
> your app out of your views and models, into separate classes? Once you
> do that, how can you test those classes without passing any Django
> objects (models, requests, etc.) into them? Answering those questions
> will be a good start. :)
>
> --
> Gary
> http://blog.extracheese.org
>