[TIP] A rare philosophical thought

[Jesse Noller]

On Fri, Aug 1, 2008 at 3:08 PM, Kumar McMillan <kumar.mcmillan at gmail.com> wrote:
> On Fri, Aug 1, 2008 at 12:28 PM, C. Titus Brown <ctb at msu.edu> wrote:
>> """
>> The Python gurus recommend unit testing to make sure code is solid.
>> That's great. If I wanted to write dozens of lines of boilerplate
>> code in order to make sure stuff worked, I'd have stuck with C++. I want
>> to write less code and be confident in the belief that that code is
>> correct and error free.
>> """
>
> Whenever I see this kind of resistance I've noticed there is often a
> misinterpretation of what "automated testing" really means.  I have
> asked developers who do not write automated tests how they guarantee
> that a feature works.  The good developers respond with: "well, I add
> the feature then I fire up the app and verify that it works.  Duh!"
>
> This is a great point of entry for helping developers to understand
> automated testing.  What they are already doing is *the same exact
> thing* as automated testing except it is manual and not automated.
> The real problem lies in that it often doesn't make sense to add
> automated tests.  That is, the app may be too small and manual testing
> could be as simple as loading up a single page then submitting a form
> and verifying the success page.  Done.  You've tested your entire app
> and you know it works.
>
> Why automated testing then?  The answer (as we all know) is in
> scalability.  That's the point to stress when trying to help a
> developer understand the usefulness of automated testing.  OK, so what
> happens when you have a sequential form that is 8 pages long?  I.E. to
> manually test this you have to start at the beginning *every time*.
> For me, this was exactly when the concept of automated testing
> clicked.  I had been working on a dating / matchmaking site and the
> profile signup was indeed about 8 or 10 form pages and there was a
> very abstract base component that drove every single page of the form.
>  So if I made a bad change to the base it would break every page of
> the form in confusing ways.  I developed this app entirely without
> automated tests and had to test the app by manually going through the
> form one page at a time!  The result of course was many subtle bugs in
> the combinations of form elements that I was too lazy to test.
>
> But even after a developer understands this concept there is still a
> problem of discipline.  If you start an app that has 2 pages of forms
> and that app grows 4 pages of forms then you will probably admit to
> yourself "OK, I should start writing some tests now."  Unfortunately,
> for the typical software development team, it is too late to start
> writing tests at this point.  With typical pressures from the business
> (MAKE MONEY NOW!) it is almost always too hard to sell the fact that
> you need to HALT new development for a month while you write automated
> tests to cover ALL the use cases you have implemented thus far.
>
> In conclusion, it is my belief that any app to grow from a prototype
> into some kind of "stable" tool is doomed to fail if no automated
> tests exist at that point of maturity.
>
> Then again, the value of manual testing should not be underestimated!
>
> RE: the original boilerplate argument, don't forget that the average
> developer writes BAD TESTS.  Especially when you are prototyping an
> app and you know you *should* write tests but you don't know how the
> app is going to be implemented this is when you often write a
> meaningless test.  Instantiating an object is not a very useful test!
> This type of testing is only going to provide more fuel for the
> anti-automated-testing developers out there, not to mention give you
> and your team a sense of false confidence.  To remedy this, I would
> advise any developer I see who writes these kind of tests to *stop*
> writing tests for a minute and start manually testing his/her code
> instead to gain an understanding of what it means to verify a use
> case.  Even unit testing is about verifying a use case -- you have an
> object or function [the unit] and another piece of code wants to use
> it.  Along those lines, doctest provides another great entry point for
> developers to understand the concept of automated testing.  So the
> second point is you have to become a "user" to write better tests.
> Luckily, this is naturally a part of software development.
>
> Kumar
>
> PS. Don't forget to blame Titus for this rant ;)
>

Dang. This is too long to fit on a t-shirt, otherwise I would :)

-jesse