<div dir="ltr">A couple points:<div><br></div><div>If you're using a virtualenv you have to activate prior to other commands and when you want to use it again from a fresh terminal. I really recommend using pip for the install as well. Once the khmer project is installed that way you can run the scripts just like any other command.</div>
<div><br></div><div>The '-R' option to normalize-by-median is for specifying the filename of a report file. According to what I see in your email you specify the same filename as your main input file. In my tests this causes the input file to be erased, which is a bug.</div>
<div><br></div><div>The kept sequences from normalize-by-median.py are saved to a file in the current directory with the same filename as the input but with ".keep" appended. This is likely the source of the error you see.</div>
<div><br></div><div>Here is my suggested variation:</div><div><br></div><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">```</span><br style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">
<span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">curl -O <a href="https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz" target="_blank">https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz</a><br>
</span><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">tar xzf virtualenv*<br></span><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">python2.7 virtualenv*/virtualenv.py khmerEnv<br>
</span><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">source khmerEnv/bin/activate<br></span><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">pip2 install khmer</span></div>
<div><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">cd </span><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px"> </span><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">/work/efraim/output/</span></div>
<div><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">interleave-reads.py trimmed_</span><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">R1_001.fasta trimmed_</span><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">R2_001.fasta > </span><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">trimmed_R.fasta</span><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px"><br>
</span></div><div><div style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">normalize-by-median.py -k 20 -N 4 -x 2e9 -C 20 -p trimmed_R.fasta -R trimmed_R.keep.report</div></div><div style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">
# output will be in /work/efraim/output/trimmed_R.fasta.keep</div><div><br></div><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">```</span><br></div><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px"><br>
</span></div><div><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">Cheers,</span></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jul 7, 2014 at 12:37 AM, Efraim Rodrigues <span dir="ltr"><<a href="mailto:efraimnaassom@gmail.com" target="_blank">efraimnaassom@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Okay, here it goes.</div><div><div><br></div><div>python interleave-reads.py /work/efraim/output/trimmed_R1_001.fasta /work/efraim/output/trimmed_R2_001.fasta > /work/efraim/khmer/output/trimmed_R.fasta</div>
<div><br></div><div>python normalize-by-median.py -k 20 -N 4 -x 2e9 -C 20 -p /work/efraim/khmer/output/trimmed_R.fasta -R /work/efraim/khmer/output/normalized_R.fasta</div></div><div><br></div><div>Unfortunately, I am still getting the same error. This is the output from the scripts interleave-reads.py and normalize-by-median.py:</div>
<div><br></div><div><div><br></div><div>|| This is the script 'interleave-reads.py' in khmer.</div><div>|| You are running khmer version 1.1</div><div>|| You are also using screed version 0.7</div><div>||</div><div>
|| If you use this script in a publication, please cite EACH of the following:</div><div>||</div><div>|| * MR Crusoe et al., 2014. doi: 10.6084/m9.figshare.979190</div><div>||</div><div>|| Please see the CITATION file for details.</div>
<div><br></div><div>Interleaving:</div><div><span style="white-space:pre-wrap"> </span>/work/efraim/output/trimmed_R1_001.fasta</div><div><span style="white-space:pre-wrap"> </span>/work/efraim/output/trimmed_R2_001.fasta</div>
<div>... 0 pairs</div><div>... 100000 pairs</div><div>... 200000 pairs</div><div>... 300000 pairs</div><div>... 400000 pairs</div><div>... 500000 pairs</div><div>... 600000 pairs</div><div>... 700000 pairs</div><div>... 800000 pairs</div>
<div>... 900000 pairs</div><div>... 1000000 pairs</div><div>... 1100000 pairs</div><div>... 1200000 pairs</div><div>... 1300000 pairs</div><div>... 1400000 pairs</div><div>... 1500000 pairs</div><div>... 1600000 pairs</div>
<div>... 1700000 pairs</div><div>... 1800000 pairs</div><div>... 1900000 pairs</div><div>... 2000000 pairs</div><div>... 2100000 pairs</div><div>... 2200000 pairs</div><div>... 2300000 pairs</div><div>... 2400000 pairs</div>
<div>... 2500000 pairs</div><div>... 2600000 pairs</div><div>... 2700000 pairs</div><div>... 2800000 pairs</div><div>... 2900000 pairs</div><div>... 3000000 pairs</div><div>... <a href="tel:3100000" value="+13100000" target="_blank">3100000</a> pairs</div>
<div>... 3200000 pairs</div>
<div>... 3300000 pairs</div><div>... 3400000 pairs</div><div>... 3500000 pairs</div><div>... 3600000 pairs</div><div>... 3700000 pairs</div><div>... 3800000 pairs</div><div>... 3900000 pairs</div><div>... 4000000 pairs</div>
<div>final: interleaved 4026442 pairs</div><div><br></div><div>|| This is the script 'normalize-by-median.py' in khmer.</div><div>|| You are running khmer version 1.1</div><div>|| You are also using screed version 0.7</div>
<div>||</div><div>|| If you use this script in a publication, please cite EACH of the following:</div><div>||</div><div>|| * MR Crusoe et al., 2014. doi: 10.6084/m9.figshare.979190</div><div>|| * CT Brown et al., arXiv:1203.4802 [q-bio.GN]</div>
<div>||</div><div>|| Please see the CITATION file for details.</div><div><br></div><div><br></div><div>PARAMETERS:</div><div> - kmer size = 20 <span style="white-space:pre-wrap"> </span>(-k)</div><div> - n tables = 4 <span style="white-space:pre-wrap"> </span>(-N)</div>
<div> - min tablesize = 2e+09 <span style="white-space:pre-wrap"> </span>(-x)</div><div><br></div><div>Estimated memory usage is 8e+09 bytes (n_tables x min_tablesize)</div><div>--------</div><div>Traceback (most recent call last):</div>
<div> File "normalize-by-median.py", line 5, in <module></div><div> pkg_resources.run_script('khmer==1.1', 'normalize-by-median.py')</div><div> File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 534, in run_script</div>
<div> File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 1434, in run_script</div><div> File "/home/efraim/.local/lib/python2.7/site-packages/khmer-1.1-py2.7-linux-x86_64.egg/EGG-INFO/scripts/normalize-by-median.py", line 312, in <module></div>
<div> main()</div><div> File "/home/efraim/.local/lib/python2.7/site-packages/khmer-1.1-py2.7-linux-x86_64.egg/EGG-INFO/scripts/normalize-by-median.py", line 243, in main</div><div> outfp = open(output_name, 'w')</div>
<div>IOError: [Errno 30] Read-only file system: 'normalized_R.fasta.keep'</div></div><div><br></div><div>Thank you!</div><div><br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">
On Sun, Jul 6, 2014 at 11:11 PM, Michael R. Crusoe <span dir="ltr"><<a href="mailto:mcrusoe@msu.edu" target="_blank">mcrusoe@msu.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">If you could include the exact command (with parameters) that is causing the problem and which directory you run it in that would be useful.</div>
<div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jul 7, 2014 at 12:09 AM, Efraim Rodrigues <span dir="ltr"><<a href="mailto:efraimnaassom@gmail.com" target="_blank">efraimnaassom@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Alright! Awesome! <div><br></div><div>I am sorry guys, I did not see that the newest version was the 1.11.6.</div>
<div><br></div><div>Well, 'make test' worked and the result was 'OK'. </div><div>
<br></div><div>I'm going back to the initial problem which was <span style="font-family:arial,sans-serif;font-size:12.727272033691406px">error IOError: [Errno 30] Read-only file system: . I'll try to run it again and I'll let you guys know about the result.</span></div>
<div><span style="font-family:arial,sans-serif;font-size:12.727272033691406px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:12.727272033691406px">Thanks one more time and I'm sorry about not getting the newest version.</span></div>
</div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sun, Jul 6, 2014 at 11:01 PM, Michael R. Crusoe <span dir="ltr"><<a href="mailto:mcrusoe@msu.edu" target="_blank">mcrusoe@msu.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The lack of `pip2` when installed this way is fixed in the latest virtualenv release (1.11.6):<div><br>
</div>
<div>```<br><span style="color:rgb(0,0,0)">curl -O <a href="https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz" target="_blank">https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz</a><br>
</span><span style="color:rgb(0,0,0)">tar xzf virtualenv*<br></span><span style="color:rgb(0,0,0)">python2.7 virtualenv*/virtualenv.py khmerEnv<br></span><span style="color:rgb(0,0,0)">source khmerEnv/bin/activate<br></span><span style="color:rgb(0,0,0)">pip2 --version<br>
</span><span style="color:rgb(0,0,0)">```</span></div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Sun, Jul 6, 2014 at 11:11 PM, C. Titus Brown <span dir="ltr"><<a href="mailto:ctb@msu.edu" target="_blank">ctb@msu.edu</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div>On Sun, Jul 06, 2014 at 10:09:20PM -0500, Efraim Rodrigues wrote:<br>
> When I try to run 'make test'. It looks like it calls a script which runs<br>
> pip2. I guess I cannot change it.<br>
><br>
> Oh, do not feel sorry. You guys are willing to help and that's all that<br>
> matters.<br>
<br>
</div>well, thanks ;)<br>
<br>
You can do:<br>
<br>
ln <virtualenv>/bin/pip <virtualenv>/bin/pip2<br>
<br>
where you replace <virtualenv> with the path to your virtualenv directory.<br>
<br>
cheers,<br>
--titus<br>
<div><div><br>
> On Sun, Jul 6, 2014 at 10:01 PM, C. Titus Brown <<a href="mailto:ctb@msu.edu" target="_blank">ctb@msu.edu</a>> wrote:<br>
><br>
> > Just try pip (not pip2). Sorry for all the mess :(<br>
> ><br>
> > On Jul 6, 2014, at 7:53 PM, Efraim Rodrigues <<a href="mailto:efraimnaassom@gmail.com" target="_blank">efraimnaassom@gmail.com</a>><br>
> > wrote:<br>
> ><br>
> > Hi. Thanks for your response.<br>
> ><br>
> > It looks like the we are supposed to choose the version of the virtualenv.<br>
> > I found all the versions here<br>
> > <a href="https://pypi.python.org/packages/source/v/virtualenv/" target="_blank">https://pypi.python.org/packages/source/v/virtualenv/</a> so I am followed<br>
> > these steps using the newest version of the virtualen, in this case, I<br>
> > believe that is,<br>
> > <a href="https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.tar.gz" target="_blank">https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.tar.gz</a><br>
> ><br>
> > Anyway, I ran all the tests and it still returns the same problem.<br>
> ><br>
> > make test<br>
> > pip2 install --user nose || pip2 install nose<br>
> > /bin/sh: pip2: command not found<br>
> > /bin/sh: pip2: command not found<br>
> > make: *** [test] Error 127<br>
> ><br>
> > In fact, there is no pip2 inside khmerEnv. I can just see these files:<br>
> ><br>
> > activate activate.fish easy_install pip python<br>
> > python2.7<br>
> > activate.csh activate_this.py easy_install-2.7 pip-2.7 python2<br>
> ><br>
> > Should I try another version of virutalenv or perhaps another similar tool?<br>
> ><br>
> > Thank y'all for trying to help me.<br>
> ><br>
> ><br>
> > On Sun, Jul 6, 2014 at 5:25 PM, C. Titus Brown <<a href="mailto:ctb@msu.edu" target="_blank">ctb@msu.edu</a>> wrote:<br>
> ><br>
> >> On Sat, Jul 05, 2014 at 03:01:47PM -0500, Efraim Rodrigues wrote:<br>
> >> > Hi everyone,<br>
> >> ><br>
> >> > Actually, when I tried to install by running 'make test', it returned<br>
> >> the<br>
> >> > following message:<br>
> >> ><br>
> >> > pip2 install --user nose || pip2 install nose<br>
> >> > /bin/sh: pip2: command not found<br>
> >> > /bin/sh: pip2: command not found<br>
> >> ><br>
> >> ><br>
> >> > Even though we have python/2.7 installed, It looks like my system has<br>
> >> > python/2.6 as default. Hence, in order to install 'khmer', I followed<br>
> >> these<br>
> >> > steps:<br>
> >> ><br>
> >> > module load python/2.7<br>
> >> ><br>
> >> > python setup.py install --user<br>
> >> ><br>
> >> > Apparently, it worked to install it locally, however I'm not sure about<br>
> >> any<br>
> >> > additional file or module.<br>
> >><br>
> >> OK, I would suggest running the tests inside of virtualenv. try:<br>
> >><br>
> >> curl -O<br>
> >> <a href="https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.x.y.tar.gz" target="_blank">https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.x.y.tar.gz</a><br>
> >> tar xzf virtualenv*<br>
> >> cd virtualenv-*; python2.7 virtualenv.py ../khmerEnv; cd ..<br>
> >> source khmerEnv/bin/activate<br>
> >><br>
> >> and then 'make test'.<br>
> >><br>
> >> thanks!<br>
> >> --titus<br>
> >><br>
> >><br>
> ><br>
<br>
</div></div><div>--<br>
C. Titus Brown, <a href="mailto:ctb@msu.edu" target="_blank">ctb@msu.edu</a><br>
<br>
</div></div></div><div><div><div>_______________________________________________<br>
khmer mailing list<br>
<a href="mailto:khmer@lists.idyll.org" target="_blank">khmer@lists.idyll.org</a><br>
<a href="http://lists.idyll.org/listinfo/khmer" target="_blank">http://lists.idyll.org/listinfo/khmer</a><br>
</div></div></div></blockquote></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><span style="font-family:'courier new',monospace;font-size:small">Michael R. Crusoe: Programmer & Bioinformatician </span><a href="mailto:mcrusoe@msu.edu" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">mcrusoe@msu.edu</a><br style="font-family:'courier new',monospace;font-size:small">
<span style="font-family:'courier new',monospace;font-size:small"> @ the Genomics, Evolution, and Development lab; Michigan State U</span><br style="font-family:'courier new',monospace;font-size:small"><a href="http://ged.msu.edu/" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">http://ged.msu.edu/</a><span style="font-family:'courier new',monospace;font-size:small"> </span><a href="http://orcid.org/0000-0002-2961-9670" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">http://orcid.org/0000-0002-2961-9670</a><span style="font-family:'courier new',monospace;font-size:small"> </span><a href="http://twitter.com/biocrusoe" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">@biocrusoe</a><br>
</div>
</font></span></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><span style="font-family:'courier new',monospace;font-size:small">Michael R. Crusoe: Programmer & Bioinformatician </span><a href="mailto:mcrusoe@msu.edu" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">mcrusoe@msu.edu</a><br style="font-family:'courier new',monospace;font-size:small">
<span style="font-family:'courier new',monospace;font-size:small"> @ the Genomics, Evolution, and Development lab; Michigan State U</span><br style="font-family:'courier new',monospace;font-size:small"><a href="http://ged.msu.edu/" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">http://ged.msu.edu/</a><span style="font-family:'courier new',monospace;font-size:small"> </span><a href="http://orcid.org/0000-0002-2961-9670" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">http://orcid.org/0000-0002-2961-9670</a><span style="font-family:'courier new',monospace;font-size:small"> </span><a href="http://twitter.com/biocrusoe" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">@biocrusoe</a><br>
</div>
</div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><span style="font-family:'courier new',monospace;font-size:small">Michael R. Crusoe: Programmer & Bioinformatician </span><a href="mailto:mcrusoe@msu.edu" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">mcrusoe@msu.edu</a><br style="font-family:'courier new',monospace;font-size:small">
<span style="font-family:'courier new',monospace;font-size:small"> @ the Genomics, Evolution, and Development lab; Michigan State U</span><br style="font-family:'courier new',monospace;font-size:small"><a href="http://ged.msu.edu/" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">http://ged.msu.edu/</a><span style="font-family:'courier new',monospace;font-size:small"> </span><a href="http://orcid.org/0000-0002-2961-9670" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">http://orcid.org/0000-0002-2961-9670</a><span style="font-family:'courier new',monospace;font-size:small"> </span><a href="http://twitter.com/biocrusoe" style="color:rgb(17,85,204);font-family:'courier new',monospace;font-size:small" target="_blank">@biocrusoe</a><br>
</div>
</div>