[TIP] testing: why bother?
michael at voidspace.org.uk
Wed Mar 23 09:38:17 PDT 2011
On 23/03/2011 16:26, Michael Foord wrote:
> On 23/03/2011 15:37, C. Titus Brown wrote:
>> Also I don't know of any actual evidence that TDD or agile actually
>> programming speed, ability, thought, or anything else, over other
>> methods of
>> specification (more formal ones, for example). Anecdotes need not apply.
These studies are the best I'm aware of on benefits of TDD:
The study compared 4 projects, at Microsoft and IBM that used TDD with
similar projects that did not use TDD.
The pre-release defect density of the four products, measured as defects
per thousand lines of code, decreased between 40% and 90% relative to
the projects that did not use TDD. The teams' management reported
subjectively a 15–35% increase in initial development time for the teams
using TDD, though the teams agreed that this was offset by reduced
Which I believe refers to a meta-paper incorporating this study:
The results summarized in Figure 5
indicate that while the development of both the systems utilizing
TDD took extra time upfront the resulting quality was higher than
teams that adopted a non-TDD approach by an order of at least two
The microsoft empirical study is interesting because they have large
scale development with varying practices, and are in a position to
compare those practises across projects.
> Well sure, but unless you have evidence to the contrary (anecdotes
> need not apply) that's entirely moot.
>>> I have actually been thinking recently that if I were to have the
>>> opportunity to teach programming to a beginner then I would like to
>>> try starting with TDD (& probably also VCS) from the very beginning. I
>>> have recently watched a couple of novice programmers suffer through
>>> their formal instruction, largely because they were always piling
>>> uncertainty upon uncertainty. When doing assignments, they would add
>>> some code that they thought worked, then some more code that they
>>> thought worked, then some more, and then after a few hours of
>>> difficult thinking and coding they would try to run the code and then
>>> see that it doesn't work and then be very upset.
>>> My hypotheses are that if they were taught to do TDD, this would be
>>> much less of a problem, and that the relatively rigid discipline of
>>> TDD makes it teachable even to programming novices.
>>> I guess this is wandering rather far off topic, but I would be very
>>> interested to hear from anyone who has tried this.
>> I'm not a big fan of TDD so I may not be doing this with conviction, but
>> I have at least tried a little bit ;).
>> Lest anyone think I'm negative on the whole concept, I'm not! I am just
>> confused about how to do a good job of inculcating testing; coming
>> from my
>> background, I thought it would be easier to put into effect, but it
>> been so far. I'd love to figure out how to do it. And how to measure the
>> effect so that I can prove I've done it.
May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
More information about the testing-in-python