[TIP] why you should distribute tests with your application / module

Grig Gheorghiu grig at gheorghiu.net
Wed Sep 17 12:16:13 PDT 2008

--- On Wed, 9/17/08, Bob Clancy <bob.clancy at verizon.net> wrote:

> From: Bob Clancy <bob.clancy at verizon.net>
> On Wed, Sep 17, 2008 at  2:38 PM, Jesse Noller wrote:
> > Sure, if he wants to start a holy war - the
> definitions he's using are 
> > fairly developer centric, rather than testing group
> specific.
> I surely doubt it will go that far!  ;-)
> I'd like to understand what terminology is in common
> use.  I'm serving 
> as a technical reviewer for someone's upcoming book on
> Agile Acceptance 
> Testing.  He follows the standard "get customers and
> developers (and 
> analysts if you have them) together in a room for
> story-writing 
> workshop" sort of method for defining testcases as
> examples, that could 
> then be added to a tool like FIT and used as Acceptance
> tests.  In my 
> practice, the most I've ever had was a product manager
> serving as a 
> customer proxy.  I have a tendency to want to define tests
> that are 
> meaninful at a higher level of workflow than a programmer
> might normally 
> be concerned with, but that might not really be useful to
> customers. 
> Who has those??? ;-)  Anyway, I think Titus convinced me to
> call a spade 
> a spade as doing so will reduce confusion down the road.

I personally like Brian Marick's distinction between customer-facing tests and code-facing tests. Here's a fragment from a blog post of mine from a couple of years ago [1]:

"Customer-facing tests are high-level tests expressed in the business domain language, and they are created (ideally) through the collaboration of customers, business analysts, testers and developers. When a customer-facing test passes, it gives the customer a warm fuzzy feeling that the application does what it's supposed to do. Customer-facing tests are usually called acceptance tests. They can operate at the business logic level (in which case, at least in agile environments, they're usually created in executed with tools such as Fit or FitNesse), or at the GUI level (in which case a variety of tools can be used; for Web application, a combination of twill and Selenium will usually do the trick).

Code-facing tests are lower-level tests expressed in the language of the programmers. They deal with the nitty-gritty of the application, and when they pass, they give developers a warm fuzzy feeling that their code does what they intended it to do, and that they didn't break any existing code when they refactored it. Unit tests are a prime example of code-facing tests."

Note the use of the work 'ideally'. In reality, as Bob says, it's very hard to have that meeting of the minds between testers, developers and customer representatives such as business analysts, where they all collaborate in writing acceptance tests.

> I still think his distinction on what's a regression
> test might be a 
> little artificial though!

I use regression test to mean a suite of tests that have passed in the previous release of the software. A test failing in the new release indicates a regression.


[1] http://agiletesting.blogspot.com/2006/04/should-acceptance-tests-be-included-in.html

More information about the testing-in-python mailing list