[TIP] Result protocol / data content

[laidler at stsci.edu]

>On Mon, Apr 13, 2009 at 9:16 PM, Doug Philips <dgou at mac.com> wrote:
>
>> It seems we have two levels of data here:
>> -- Data having to do with an individual test.
>> -- Data having to do with the aggregate of some number of individual
>> tests.
>
>no, we have one thing representing a container which contains enough
>information to identify, debug and otherwise provide a fantastic
>amount of information about a group of tests executed.

Maybe I missed it in the flood of email, but could you clarify your
motivation for wanting to define the result protocol as inherently
describing an aggregate of tests, rather than an individual test?
What problem are you trying to solve?

I get that a single case can be represented in this protocol as
the special case for which N=1, but an aggregate seems to me to be 
a peculiar choice of building block.

What are the advantages of defining an aggregate as the basic
level; rather than defining a single test result as the basic level,
and letting downstream critters do the aggregating? Or is the idea
that you know you want to aggregate so why define two protocols
(one for a single case and one for a bunch) when the former can 
be a special case of the latter?

If it is an aggregate, are there issues with size or complexity or
levels of nesting, that would impose a desirable or practical upper
limit on the size and structure of the aggregate?

>
>> --everything-- else should be optional, not included if not needed.
>
>Individual implementations can exclude or include information (and
>extend for more) as needed. I don't functionally see what the problem
>is in defining a hierarchy of data.

>
>> so I'm not sure it is needed. Most of the additional
>> data I'm OK with, but I don't see the need for all this nesting
>> brouhaha, seems over engineered. I can see one level of nesting for
>> the attributes associated with each test. Additional nesting within
>> should be implementation-extension. Really, if I want stdout, stderr,
>> trace-foo, whatever, do I really need a separate container for it?
>
>Error: Can not parse. I don't see how *minor* nesting, in order to
>compartmentalize data which is alike, or will be repeated is a *bad*
>thing. I simply do not grok what is wrong with:
>
>run data: { test_cases: {} }
>
>The test cases are a result of the run, ergo, they are children of a
>given run. Data about the test is a child of the test. This seems like
>a natural fit, and pretty clear.

Mark and I have gone round in circles a few times on whether the 
testrun should be the parent of a test, or an identifying attribute
of the test.