[TIP] testing: why bother?

[Michael Foord]

On 23/03/2011 16:26, Michael Foord wrote:
> On 23/03/2011 15:37, C. Titus Brown wrote:
>> [snip...]
>> Also I don't know of any actual evidence that TDD or agile actually 
>> improves
>> programming speed, ability, thought, or anything else, over other 
>> methods of
>> specification (more formal ones, for example). Anecdotes need not apply.
>>
>

These studies are the best I'm aware of on benefits of TDD:

* http://www.infoq.com/news/2009/03/TDD-Improves-Quality

The study compared 4 projects, at Microsoft and IBM that used TDD with 
similar projects that did not use TDD.

The pre-release defect density of the four products, measured as defects 
per thousand lines of code, decreased between 40% and 90% relative to 
the projects that did not use TDD. The teams' management reported 
subjectively a 15–35% increase in initial development time for the teams 
using TDD, though the teams agreed that this was offset by reduced 
maintenance costs.

Which I believe refers to a meta-paper incorporating this study:

* http://research.microsoft.com/en-us/groups/ese/fp17288-bhat.pdf

The results summarized in Figure 5
indicate that while the development of both the systems utilizing
TDD took extra time upfront the resulting quality was higher than
teams that adopted a non-TDD approach by an order of at least two
times.

The microsoft empirical study is interesting because they have large 
scale development with varying practices, and are in a position to 
compare those practises across projects.

Michael

> Well sure, but unless you have evidence to the contrary (anecdotes 
> need not apply) that's entirely moot.
>
> Michael
>
>>> I have actually been thinking recently that if I were to have the
>>> opportunity to teach programming to a beginner then I would like to
>>> try starting with TDD (& probably also VCS) from the very beginning. I
>>> have recently watched a couple of novice programmers suffer through
>>> their formal instruction, largely because they were always piling
>>> uncertainty upon uncertainty. When doing assignments, they would add
>>> some code that they thought worked, then some more code that they
>>> thought worked, then some more, and then after a few hours of
>>> difficult thinking and coding they would try to run the code and then
>>> see that it doesn't work and then be very upset.
>>>
>>> My hypotheses are that if they were taught to do TDD, this would be
>>> much less of a problem, and that the relatively rigid discipline of
>>> TDD makes it teachable even to programming novices.
>>>
>>> I guess this is wandering rather far off topic, but I would be very
>>> interested to hear from anyone who has tried this.
>> I'm not a big fan of TDD so I may not be doing this with conviction, but
>> I have at least tried a little bit ;).
>>
>> Lest anyone think I'm negative on the whole concept, I'm not! I am just
>> confused about how to do a good job of inculcating testing; coming 
>> from my
>> background, I thought it would be easier to put into effect, but it 
>> hasn't
>> been so far. I'd love to figure out how to do it. And how to measure the
>> effect so that I can prove I've done it.
>>
>> --titus
>
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html