[bip] Reproducible research

[Nathan Harmston]

Couple of papers/comment that I think would be relevant to this thread.

The trouble with replication
http://www.nature.com/nature/journal/v442/n7101/full/442344a.html

Peer usage versus peer review
http://www.bmj.com/cgi/content/extract/335/7617/451

Regarding Hypothesis Generation..... I agree on the data snooping bias
problems but in todays world of over/mis used /badly analysed microarrays
and other high throughput experiments what other way is there. So long as
your assumptions and model and explicit and you can account for the any bias
and you have a valid approach for selecting the best hypothesis (whether its
using a simple cost function or Bayesian Model selection), but I guess its
another thread.

Andrew: I was suggesting that the journal runs an svn for both the paper, SI
and any associated code, which would be "locked" at point of publication.
This could allow access to previous versions of the paper, comments on it,
code etc. I think I didn't put that idea across too well....I was in the
middle of trying to replicate a paper with a shitty methods section and
getting very angry.





2009/3/9 Leighton Pritchard <lpritc at scri.ac.uk>
>
> Hi,
>
> On 09/03/2009 17:25, "Andrew Dalke" <dalke at dalkescientific.com> wrote:
>
> > On Mar 9, 2009, at 10:20 AM, Leighton Pritchard wrote:
> >> <pedant>
> >> No - they're observations.  No observations are reproducible in a
> >> strict
> >> sense.  Observations play an important role in Science, but they
> >> are not
> >> Science.
> >> </pedant>
> >
> > Observations alone are meaningless.
> >
> > Observations when used as tests of theory are science.
>
> Absolutely - and they're also part of science when used to generate
> hypotheses.  But in and of themselves, they are not science.  The
framework
> in which the observations are embedded is science.  I think we actually
> agree on that point.
>
> For example, I observe two mugs on my desk, but that is not science.
> Assuming I don't want to do science with coffee mugs right now, of course
;)
>
> > Are the following science?
> >    - Eddington's observation of light deflection in the
> >       eclipse of 1919 as a test of Einstein's general relativity
> >
> >    - Observation of the muon, in conflict with the particle theory of
> > 1936
> >
> > (Can you infer my physics background here? :)
>
> It's coming through, but only subtly ;)
>
> > If these are not science, then what is science?
>
> They're science in the same sense that a brick is a house.  If neither of
> those two (particular) observations had been made, some other equivalent
> observation, confirming or falsifying either theory could have been, just
as
> any brick in the building I sit could have been substituted for some other
> brick.  The whole does not confer all its properties to the part; the
brick
> does not subclass the building ;)
>
> Defining Science is a tough question, but as far as I can condense it for
my
> own purposes, Science is the systematic enquiry into the behaviour and
> structure of the universe around us, by means of observation and
experiment,
> most usually through application of the Scientific Method.  Science may
also
> refer to a subdivision of this endeavour by discipline or speciality, the
> body of knowledge revealed in this way, or the social edifice of the
> practice of Science.
>
> Or a belief in the ignorance of experts, if you ask Feynman ;)
>
> L.
>
> --
> Dr Leighton Pritchard MRSC
> D131, Plant Pathology Programme, SCRI
> Errol Road, Invergowrie, Perth and Kinross, Scotland, DD2 5DA
> e:lpritc at scri.ac.uk <e%3Alpritc at scri.ac.uk>       w:
http://www.scri.ac.uk/staff/leightonpritchard
> gpg/pgp: 0xFEFC205C       tel:+44(0)1382 562731 x2405
>
>
> ______________________________________________________________________
> SCRI, Invergowrie, Dundee, DD2 5DA.
> The Scottish Crop Research Institute is a charitable company limited by
> guarantee.
> Registered in Scotland No: SC 29367.
> Recognised by the Inland Revenue as a Scottish Charity No: SC 006662.
>
>
> DISCLAIMER:
>
> This email is from the Scottish Crop Research Institute, but the views
> expressed by the sender are not necessarily the views of SCRI and its
> subsidiaries.  This email and any files transmitted with it are
> confidential
>
> to the intended recipient at the e-mail address to which it has been
> addressed.  It may not be disclosed or used by any other than that
> addressee.
> If you are not the intended recipient you are requested to preserve this
>
> confidentiality and you must not use, disclose, copy, print or rely on
> this
> e-mail in any way. Please notify postmaster at scri.ac.uk quoting the
> name of the sender and delete the email from your system.
>
> Although SCRI has taken reasonable precautions to ensure no viruses are
> present in this email, neither the Institute nor the sender accepts any
> responsibility for any viruses, and it is your responsibility to scan
> the email and the attachments (if any).
> ______________________________________________________________________
>
> _______________________________________________
> biology-in-python mailing list - bip at lists.idyll.org.
>
> See http://bio.scipy.org/ for our Wiki.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.idyll.org/pipermail/biology-in-python/attachments/20090310/b035dfde/attachment.htm