<div dir="ltr">Ok sorry for the scare, it looks like it was a transient I/O issue. Second time it ran through successfully. Thanks!<br></div><div class="gmail_extra"><br clear="all"><div><div dir="ltr"><br>--<br>Daniel S. Standage<br>Ph.D. Candidate<br>Computational Genome Science Laboratory<br>Indiana University<br></div></div>
<br><div class="gmail_quote">On Thu, Sep 18, 2014 at 9:25 AM, C. Titus Brown <span dir="ltr"><<a href="mailto:ctb@msu.edu" target="_blank">ctb@msu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">If it's not responding to kill -9, then it's an OS-level hang on I/O, most<br>
likely; that's probably not khmer's fault!<br>
<br>
If it responds to kill -9 but not Ctrl-C, then it's a threading issue,<br>
which would not be unheard of.<br>
<br>
cheers,<br>
--titus<br>
<div class="HOEnZb"><div class="h5"><br>
On Wed, Sep 17, 2014 at 04:53:09PM -0400, Daniel Standage wrote:<br>
> I don't know if this helps, but: the command is not responding to Ctrl-c or<br>
> kill -9 as root.<br>
><br>
><br>
> --<br>
> Daniel S. Standage<br>
> Ph.D. Candidate<br>
> Computational Genome Science Laboratory<br>
> Indiana University<br>
><br>
> On Wed, Sep 17, 2014 at 4:38 PM, Daniel Standage <<a href="mailto:daniel.standage@gmail.com">daniel.standage@gmail.com</a>><br>
> wrote:<br>
><br>
> > Here it is: <a href="https://gist.github.com/standage/4222de94bd695f23f673" target="_blank">https://gist.github.com/standage/4222de94bd695f23f673</a><br>
> ><br>
> > Forgot to mention, running khmer 1.1<br>
> ><br>
> > Thanks,<br>
> > Daniel<br>
> ><br>
> ><br>
> > --<br>
> > Daniel S. Standage<br>
> > Ph.D. Candidate<br>
> > Computational Genome Science Laboratory<br>
> > Indiana University<br>
> ><br>
> > On Wed, Sep 17, 2014 at 4:31 PM, C. Titus Brown <<a href="mailto:ctb@msu.edu">ctb@msu.edu</a>> wrote:<br>
> ><br>
> >> On Wed, Sep 17, 2014 at 04:27:09PM -0400, Daniel Standage wrote:<br>
> >> > Before running norm-by-median, I<br>
> >> ><br>
> >> > - Downloaded SRA file<br>
> >> > - Used fastq-dump to create paired Fastq files<br>
> >> > - used interleave-reads to create a Fastq file in the One True Format<br>
> >> ><br>
> >> > All of the Fastq files seem to be fine i.e. none appear truncated.<br>
> >> Memory<br>
> >> > usage is remaining constant, CPU utilization is 100%, but the weird<br>
> >> thing<br>
> >> > is that as far as I can tell the norm-by-median script is complete. It<br>
> >> has<br>
> >> > processed all the input, given a final report, and all of the kept reads<br>
> >> > have been written to output: except the last read is missing and the<br>
> >> second<br>
> >> > to last read is cut off.<br>
> >><br>
> >> Sounds like a classic fencepost error :(.<br>
> >><br>
> >> Could you send me (on or off the list) the SRA commands you used and<br>
> >> the command line you ran with diginorm? Then we'll see if we can<br>
> >> replicate<br>
> >> on our own hardware.<br>
> >><br>
> >> cheers,<br>
> >> --titus<br>
> >><br>
> >> > On Wed, Sep 17, 2014 at 4:21 PM, C. Titus Brown <<a href="mailto:ctb@msu.edu">ctb@msu.edu</a>> wrote:<br>
> >> ><br>
> >> > > Hi Daniel,<br>
> >> > ><br>
> >> > > sounds like an infinite loop of some sort :(.<br>
> >> > ><br>
> >> > > A few questions --<br>
> >> > ><br>
> >> > > What version of khmer are you using?<br>
> >> > ><br>
> >> > > Have you run the reads file through any other software? I'm worried<br>
> >> > > that the file is truncated in some way.<br>
> >> > ><br>
> >> > > Do you know how far through your reads file it's gotten?<br>
> >> > ><br>
> >> > > Is memory usage increasing or remaining constant?<br>
> >> > ><br>
> >> > > thanks,<br>
> >> > > --titus<br>
> >> > ><br>
> >> > > On Wed, Sep 17, 2014 at 04:16:37PM -0400, Daniel Standage wrote:<br>
> >> > > > Hi all,<br>
> >> > > ><br>
> >> > > > I am seeing some strange behavior running normalize-by-median.py.<br>
> >> The<br>
> >> > > > program seemed to complete successfully after 30-45 minutes, but<br>
> >> then it<br>
> >> > > > just hung there. It's now been at least 90 minutes and it's<br>
> >> continuing to<br>
> >> > > > hang. The output file seems to contain all the data except the last<br>
> >> > > record,<br>
> >> > > > and the second-to-last record is cut off.<br>
> >> > > ><br>
> >> > > > (khmer-env)[standage@bggnomic qc] tail SRR494178_int.fastq.keep<br>
> >> > > > +<br>
> >> > > ><br>
> >> > ><br>
> >> GBGED>>E##################################################################################<br>
> >> > > > @SRR494178.12090255/1<br>
> >> > > ><br>
> >> > ><br>
> >> TCGAGGACNACCTTTTGACCCTTCTGCAACCTTTGAATTTCAGACATCAAACTCTCCCTCTGTCGTGTCTCCNNCAATGATGGGTCGGGC<br>
> >> > > > +<br>
> >> > > ><br>
> >> > ><br>
> >> IIIIIGGG#GGGGGGIIIIIIIIIIIIIIIIGIHIIIIIGIIIIIIIIIIIIIHIIIIHIEGHHIFIHII=?##?;9>>;IGBFFGBD8G<br>
> >> > > > @SRR494178.12090255/2<br>
> >> > > ><br>
> >> > ><br>
> >> GATTCCGTCACCGAGGAGTATCCGTTGCCGAGGTTGTGCGTCTGTCGAACCTGGCCGTTCTTTTTGACCGTGTAGGTGCCGCCGTTGATC<br>
> >> > > > +<br>
> >> > > > IIIIIIHIIIIIIIIIBIHHIIIGIIIIIII(khmer-env)[standage@bggnomic qc]<br>
> >> > > ><br>
> >> > > > Any ideas as to what could be causing this?<br>
> >> > > ><br>
> >> > > > Thanks,<br>
> >> > > > Daniel<br>
> >> > > ><br>
> >> > > > PS.<br>
> >> > > ><br>
> >> > > > - OS: Fedora 20 with lots o RAM (100s of GB)<br>
> >> > > > - Command: normalize-by-median.py -k 17 -p -N 4 -x 8e9<br>
> >> > > > SRR494178_int.fastq<br>
> >> > > > - Data: <a href="http://www.ncbi.nlm.nih.gov/sra/?term=SRR494178" target="_blank">http://www.ncbi.nlm.nih.gov/sra/?term=SRR494178</a><br>
> >> > > ><br>
> >> > > ><br>
> >> > > > --<br>
> >> > > > Daniel S. Standage<br>
> >> > > > Ph.D. Candidate<br>
> >> > > > Computational Genome Science Laboratory<br>
> >> > > > Indiana University<br>
> >> > ><br>
> >> > > > _______________________________________________<br>
> >> > > > khmer mailing list<br>
> >> > ><br>
> >> > > --<br>
> >> > > C. Titus Brown, <a href="mailto:ctb@msu.edu">ctb@msu.edu</a><br>
> >> > ><br>
> >><br>
> >> --<br>
> >> C. Titus Brown, <a href="mailto:ctb@msu.edu">ctb@msu.edu</a><br>
> >><br>
> ><br>
> ><br>
<br>
--<br>
C. Titus Brown, <a href="mailto:ctb@msu.edu">ctb@msu.edu</a><br>
</div></div></blockquote></div><br></div>