[TIP] Pandokia - more info

[Mark Sienkiewicz]

Douglas Philips wrote:
>> As promised, I've put some more info about Pandokia, our test  
>> management
>> & reporting system, on the web at http://stsdas.stsci.edu/pandokia/
>>     
>
> Woot!
> Still letting it all sink in. Hard not remembered that the results  
> format was multiline. Not sure if that is Simple enough in the  
> Titusian sense, but we'll find out that soon enough. 

The format is driven by the requirements:

- a record contains multiple named fields
- most of the fields are optional
- you do not know how many fields or all of their names until run time
- some fields contain multi-line values (sometimes thousands of lines)

I think it is a very simple format both to write and to read, but if you 
think of a simplification, I'd like to hear it.  I'm a big fan of RFC 
1925, especially section 2 (12):

   (12) In protocol design, perfection has been reached not when there
        is nothing left to add, but when there is nothing left to take
        away.

> I'm not so  
> sanguine about using a database for storing the results, but that's a  
> tool chain thing, the data formats are what I'm most interested in.
>   

I'm not that quick to look for a database either.  I've lost count of 
how many times I've passed on even evaluating a free software package 
just because it needed a working mysql to install it.

sqlite is what really makes the database tolerable.  There is no server 
and no configuration.  The database engine runs in your process and uses 
a plain file to store the data.  In that respect, a user doesn't even 
need to know that there is a database there.  (Well, if they have a 
python with a working sqlite3 module, that is...)

When you see the queries we make, the database makes more sense.  I 
might have been able to kludge it all into berkeley dbm, but that would 
have added significantly to the complexity and the development time.  
And I would be wondering what happens next month when my database goes 
over 2 GB.

I also expect that pandokia will be adaptable to other database 
engines.  This may be an advantage to somebody who has large user base.

b.t.w.  Firefox 3 uses sqlite internally, so there is a good chance that 
you've already seen how unobtrusive it is.

Mark Sienkiewicz
STScI

p.s. Hi everybody.  I've been reading this "low volume" list for about a 
week. :)