dgou at mac.com
Fri Apr 10 21:39:19 PDT 2009
On or about 2009 Apr 10, at 11:21 PM, Jesse Noller indited:
> In my world, the test doesn't need to know *anything* about the world
> around it, only that it should run, and here's some data about the
> thing it should run against. It runs, it can log as much as it likes
> (I'm watching it's stdout, stderr and anything it adds to the
> surrounding directory) and then exit 0 or 1 or XX depending on if it
> passed or failed.
never said that it needed to know anything about the world around
itself, though I find the prospect of "watching the surrounding
directory" rather odd. Is that where these charts and graphs and what
not come from?
Personally my interest is in something much simpler. it isn't the test
that I want to stream back, it is the test running infrastructure that
will keep track of what is going on and reporting back on things. But
even that can get wedged/confused (esp. if it is a simple thing
without much smarts), so I don't want the wire level protocol to be
any more fragile than it has to be. As when I say streaming, I don't
necessarily mean keeping a connection open, it could be done RESTfully
for that matter.
> Yes, this puts more of the burden on the framework writer - you have
> to allow for plugins, you have to realize some tests are dumb, and so
> on, but it's complexity you only need to do once.
> No, must be good because it has some nice attributes, like cleanliness
> and other things - again, it's an idea, and not my first when it comes
> to results formatting (I really like human readable). Ultimately; I do
> not want the tests to have to know what my framework "wants" - I want
> it to run, and somehow provide me a result. I'll handle result
Never meant to mislead you that it was the tests that had to know
anything about the surrounding framework.
As to results formatting, I'm not sure how "human readable" jibes with
charts and graphs and arbitrary blobs of data in json/yaml/protocol-
> Sorry my rudimentary vaporous design specification doesn't fit your
> needs. *sad*
It doesn't seem that your vapour has condensed enough yet, since your
position seems to still be shifting/settling. We'll see.
Personally, I'd just like to stay focused on getting the test
reporting wire level protocol down. That should be an orthogonal
concern, esp. orthogonal to such issues as 'continuously open socket'
or "transport via REST" or "file on a USB memory stick", but lots of
other dimensions as well.
More information about the testing-in-python