[TIP] test definitions
jnoller at gmail.com
Wed Sep 17 13:00:34 PDT 2008
On Wed, Sep 17, 2008 at 3:42 PM, C. Titus Brown <ctb at msu.edu> wrote:
> -> Acceptance Test: These normally comprise a large number of your tests
> -> in an organization. Acceptance tests prove that the specific
> -> component/feature is sane in the context of the fully deployed product
> -> - you might require these to be fully developed, executed and passing
> -> before a specific component or feature is merged to trunk. Acceptance
> -> tests prove that the feature/component works as intended (not
> -> programmed). They should be short in execution time.
> Interesting -- I've not hear this definition before, in agile
> terminology at least. I've heard "integration testing" used for this,
> though, as in "continuous integration"...
> -> Functional Tests: Functional tests are "larger" and should test as
> -> much of the functionality of the feature/component as possible, they
> -> should also test with an eye towards other parts of the product and
> -> system (e.g. integration). Functional tests should be as expansive and
> -> detailed as possible. These can also be called Regression tests.
See, that's the rub (methinks) I'm not coming from the agile part of
town - yes, acceptance tests can also be called integration tests,
given they're the level-up from smoke tests where you actually have an
installed system, and perform testing against the system. This is a
slightly altered version of continuous integration. With Agile stuff.
Acceptance/Integration tests are the "low bar" to jump over before you
can merge to trunk.
It's simply a matter of where you're coming from - for example, take
this not-quite-agile approach to software development:
Feature A is decided on (say, add a web gui on a DB)
Feature A's functionality (e.g. requirements) are defined by Dev,
Business and QA
Dev and QA begin writing Technical Specifications/Design documents as
well as test documents (strategy/test cases)
A branch is created for Feature A
Dev, QA (automation) go off and write "all the code" - this includes
the core code, unit tests, acceptance tests, some amount of functional
Before Feature A can be merged, all of the agreed-upon acceptance
tests must pass, the Design docs/test plans/etc all must be finalized.
Feature A is merged to trunk
All functional tests, in addition to all the other tests must be
passing before the release containing Feature A can ship.
Trunk is always ship-able
All told, you've accomplished a deep-dive into the component - you've
got all the documentation you need and you've "shipped" (to trunk) the
completed feature including all of the nice tests you've written.
More information about the testing-in-python