[bip] Bioinformatics Programming Language Shootout, Python performance poopoo'd

[Paulo Nuin]

Titus Brown wrote:

> On Fri, Feb 08, 2008 at 04:28:30PM +0100, Andrew Dalke wrote:
> -> On Feb 8, 2008, at 3:23 PM, Bruce Southey wrote:
> -> > I strong concur with Andrew that we need to focus solely
> -> > on the merit of the paper not the journal or review process.
> ->    ...
> -> > Really the problem exists at the author level because
> -> > you can not expect the reviewers to know all languages.
> -> 
> -> Hmm.  The reviewers don't need to know all the languages.  I don't.   
> -> But I know enough about some of the languages to tell there's  
> -> methodological errors.
> -> 
> -> If the reviewers can't review the content of the paper, then what's  
> -> the point of having reviewers?
> -> 
> -> So I think the reviewers do share some of the blame with the authors.
>
> We could also contact the authors directly and suggest that they
> withdraw the paper.  I don't have a good sense yet for whether or not
> the errors are that significant (although the Perl folk seem upset by
> the publication, too).
>   
I don't think they would accept to withdraw the paper, especially after 
being reviewed and accepted by the editor. I guess the main problem is 
the reproducibility of the results. We have no idea, except for the 
alignment, what files were used as input. The directions given to get 
similar files are not correct or were not explained adequately in the 
paper. This is where I think the peer review process is at blame, for 
not checking the code (at least testing it to see if it works) and the 
methodology used in obtaining input files. I believe not testing 
programs is quite common in peer-reviewing as I had papers rejected that 
the reviewers never touched the software/code.

Apart from blogging about it, one idea would be to submit a response 
article, if not in BMC Bioinformatics, somewhere else.

Paulo