[khmer] normalize-by-median.py - IOError: [Errno 30] Read-only file system:

Efraim Rodrigues efraimnaassom at gmail.com
Mon Jul 7 10:58:10 PDT 2014 

Alright! It worked! Thanks to everyone for your time.

I am just curious about a point. I did not quite understand the point about
my main input file that is being erased.  What I understood is that the
report file is going to be generated from */work/efraim/khmer/output/*
*normalized_R.fasta*, then *normalized_R.fasta.keep*. On the other hand,
the output file would be generated from
/work/efraim/khmer/output/*trimmed_R.fasta
*which would result /work/efraim/khmer/output/*trimmed_R.fasta.keep* without
any conflict, right?

One more time, thanks for your support.

Regards,
Efraim Rodrigues.


On Mon, Jul 7, 2014 at 12:00 AM, Michael R. Crusoe <mcrusoe at msu.edu> wrote:

> A couple points:
>
> If you're using a virtualenv you have to activate prior to other commands
> and when you want to use it again from a fresh terminal. I really recommend
> using pip for the install as well. Once the khmer project is installed that
> way you can run the scripts just like any other command.
>
> The '-R' option to normalize-by-median is for specifying the filename of a
> report file. According to what I see in your email you specify the same
> filename as your main input file. In my tests this causes the input file to
> be erased, which is a bug.
>
> The kept sequences from normalize-by-median.py are saved to a file in the
> current directory with the same filename as the input but with ".keep"
> appended. This is likely the source of the error you see.
>
> Here is my suggested variation:
>
> ```
> curl -O
> https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz
> tar xzf virtualenv*
> python2.7 virtualenv*/virtualenv.py khmerEnv
> source khmerEnv/bin/activate
> pip2 install khmer
> cd  /work/efraim/output/
> interleave-reads.py trimmed_R1_001.fasta trimmed_R2_001.fasta >
> trimmed_R.fasta
> normalize-by-median.py -k 20 -N 4 -x 2e9 -C 20 -p trimmed_R.fasta -R
> trimmed_R.keep.report
> # output will be in /work/efraim/output/trimmed_R.fasta.keep
>
> ```
>
> Cheers,
>
>
> On Mon, Jul 7, 2014 at 12:37 AM, Efraim Rodrigues <efraimnaassom at gmail.com
> > wrote:
>
>> Okay, here it goes.
>>
>> python interleave-reads.py /work/efraim/output/trimmed_R1_001.fasta
>> /work/efraim/output/trimmed_R2_001.fasta >
>> /work/efraim/khmer/output/trimmed_R.fasta
>>
>> python normalize-by-median.py -k 20 -N 4 -x 2e9 -C 20 -p
>> /work/efraim/khmer/output/trimmed_R.fasta -R
>> /work/efraim/khmer/output/normalized_R.fasta
>>
>> Unfortunately, I am still getting the same error. This is the output from
>> the scripts interleave-reads.py and normalize-by-median.py:
>>
>>
>> || This is the script 'interleave-reads.py' in khmer.
>> || You are running khmer version 1.1
>> || You are also using screed version 0.7
>> ||
>> || If you use this script in a publication, please cite EACH of the
>> following:
>> ||
>> ||   * MR Crusoe et al., 2014. doi: 10.6084/m9.figshare.979190
>> ||
>> || Please see the CITATION file for details.
>>
>> Interleaving:
>> /work/efraim/output/trimmed_R1_001.fasta
>> /work/efraim/output/trimmed_R2_001.fasta
>> ... 0 pairs
>> ... 100000 pairs
>> ... 200000 pairs
>> ... 300000 pairs
>> ... 400000 pairs
>> ... 500000 pairs
>> ... 600000 pairs
>> ... 700000 pairs
>> ... 800000 pairs
>> ... 900000 pairs
>> ... 1000000 pairs
>> ... 1100000 pairs
>> ... 1200000 pairs
>> ... 1300000 pairs
>> ... 1400000 pairs
>> ... 1500000 pairs
>> ... 1600000 pairs
>> ... 1700000 pairs
>> ... 1800000 pairs
>> ... 1900000 pairs
>> ... 2000000 pairs
>> ... 2100000 pairs
>> ... 2200000 pairs
>> ... 2300000 pairs
>> ... 2400000 pairs
>> ... 2500000 pairs
>> ... 2600000 pairs
>> ... 2700000 pairs
>> ... 2800000 pairs
>> ... 2900000 pairs
>> ... 3000000 pairs
>> ... 3100000 pairs
>> ... 3200000 pairs
>> ... 3300000 pairs
>> ... 3400000 pairs
>> ... 3500000 pairs
>> ... 3600000 pairs
>> ... 3700000 pairs
>> ... 3800000 pairs
>> ... 3900000 pairs
>> ... 4000000 pairs
>> final: interleaved 4026442 pairs
>>
>> || This is the script 'normalize-by-median.py' in khmer.
>> || You are running khmer version 1.1
>> || You are also using screed version 0.7
>> ||
>> || If you use this script in a publication, please cite EACH of the
>> following:
>> ||
>> ||   * MR Crusoe et al., 2014. doi: 10.6084/m9.figshare.979190
>> ||   * CT Brown et al., arXiv:1203.4802 [q-bio.GN]
>> ||
>> || Please see the CITATION file for details.
>>
>>
>> PARAMETERS:
>>  - kmer size =    20 (-k)
>>  - n tables =     4 (-N)
>>  - min tablesize = 2e+09 (-x)
>>
>> Estimated memory usage is 8e+09 bytes (n_tables x min_tablesize)
>> --------
>> Traceback (most recent call last):
>>   File "normalize-by-median.py", line 5, in <module>
>>     pkg_resources.run_script('khmer==1.1', 'normalize-by-median.py')
>>   File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 534, in
>> run_script
>>   File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 1434, in
>> run_script
>>   File
>> "/home/efraim/.local/lib/python2.7/site-packages/khmer-1.1-py2.7-linux-x86_64.egg/EGG-INFO/scripts/normalize-by-median.py",
>> line 312, in <module>
>>     main()
>>   File
>> "/home/efraim/.local/lib/python2.7/site-packages/khmer-1.1-py2.7-linux-x86_64.egg/EGG-INFO/scripts/normalize-by-median.py",
>> line 243, in main
>>     outfp = open(output_name, 'w')
>> IOError: [Errno 30] Read-only file system: 'normalized_R.fasta.keep'
>>
>> Thank you!
>>
>>
>>
>> On Sun, Jul 6, 2014 at 11:11 PM, Michael R. Crusoe <mcrusoe at msu.edu>
>> wrote:
>>
>>> If you could include the exact command (with parameters) that is causing
>>> the problem and which directory you run it in that would be useful.
>>>
>>>
>>> On Mon, Jul 7, 2014 at 12:09 AM, Efraim Rodrigues <
>>> efraimnaassom at gmail.com> wrote:
>>>
>>>> Alright! Awesome!
>>>>
>>>> I am sorry guys, I did not see that the newest version was the 1.11.6.
>>>>
>>>> Well, 'make test' worked and the result was 'OK'.
>>>>
>>>> I'm going back to the initial problem which was error  IOError: [Errno
>>>> 30] Read-only file system: . I'll try to run it again and I'll let you guys
>>>> know about the result.
>>>>
>>>> Thanks one more time and I'm sorry about not getting the newest version.
>>>>
>>>>
>>>> On Sun, Jul 6, 2014 at 11:01 PM, Michael R. Crusoe <mcrusoe at msu.edu>
>>>> wrote:
>>>>
>>>>> The lack of `pip2` when installed this way is fixed in the latest
>>>>> virtualenv release (1.11.6):
>>>>>
>>>>>  ```
>>>>> curl -O
>>>>> https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz
>>>>> tar xzf virtualenv*
>>>>> python2.7 virtualenv*/virtualenv.py khmerEnv
>>>>> source khmerEnv/bin/activate
>>>>> pip2 --version
>>>>> ```
>>>>>
>>>>>
>>>>> On Sun, Jul 6, 2014 at 11:11 PM, C. Titus Brown <ctb at msu.edu> wrote:
>>>>>
>>>>>> On Sun, Jul 06, 2014 at 10:09:20PM -0500, Efraim Rodrigues wrote:
>>>>>> > When I try to run 'make test'. It looks like it calls a script
>>>>>> which runs
>>>>>> > pip2. I guess I cannot change it.
>>>>>> >
>>>>>> > Oh, do not feel sorry. You guys are willing to help and that's all
>>>>>> that
>>>>>> > matters.
>>>>>>
>>>>>> well, thanks ;)
>>>>>>
>>>>>> You can do:
>>>>>>
>>>>>>         ln <virtualenv>/bin/pip <virtualenv>/bin/pip2
>>>>>>
>>>>>> where you replace <virtualenv> with the path to your virtualenv
>>>>>> directory.
>>>>>>
>>>>>> cheers,
>>>>>> --titus
>>>>>>
>>>>>> > On Sun, Jul 6, 2014 at 10:01 PM, C. Titus Brown <ctb at msu.edu>
>>>>>> wrote:
>>>>>> >
>>>>>> > > Just try pip (not pip2). Sorry for all the mess :(
>>>>>> > >
>>>>>> > > On Jul 6, 2014, at 7:53 PM, Efraim Rodrigues <
>>>>>> efraimnaassom at gmail.com>
>>>>>> > > wrote:
>>>>>> > >
>>>>>> > > Hi. Thanks for your response.
>>>>>> > >
>>>>>> > > It looks like the we are supposed to choose the version of the
>>>>>> virtualenv.
>>>>>> > > I found all the versions here
>>>>>> > > https://pypi.python.org/packages/source/v/virtualenv/ so I am
>>>>>> followed
>>>>>> > > these steps using the newest version of the virtualen, in this
>>>>>> case, I
>>>>>> > > believe that is,
>>>>>> > >
>>>>>> https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.tar.gz
>>>>>> > >
>>>>>> > > Anyway, I ran all the tests and it still returns the same problem.
>>>>>> > >
>>>>>> > >  make test
>>>>>> > > pip2 install --user nose || pip2 install nose
>>>>>> > > /bin/sh: pip2: command not found
>>>>>> > > /bin/sh: pip2: command not found
>>>>>> > > make: *** [test] Error 127
>>>>>> > >
>>>>>> > > In fact, there is no pip2 inside khmerEnv. I can just see these
>>>>>> files:
>>>>>> > >
>>>>>> > > activate      activate.fish     easy_install      pip      python
>>>>>> > > python2.7
>>>>>> > > activate.csh  activate_this.py  easy_install-2.7  pip-2.7  python2
>>>>>> > >
>>>>>> > > Should I try another version of virutalenv or perhaps another
>>>>>> similar tool?
>>>>>> > >
>>>>>> > > Thank y'all for trying to help me.
>>>>>> > >
>>>>>> > >
>>>>>> > > On Sun, Jul 6, 2014 at 5:25 PM, C. Titus Brown <ctb at msu.edu>
>>>>>> wrote:
>>>>>> > >
>>>>>> > >> On Sat, Jul 05, 2014 at 03:01:47PM -0500, Efraim Rodrigues wrote:
>>>>>> > >> > Hi everyone,
>>>>>> > >> >
>>>>>> > >> > Actually, when I tried to install by running 'make test', it
>>>>>> returned
>>>>>> > >> the
>>>>>> > >> > following message:
>>>>>> > >> >
>>>>>> > >> > pip2 install --user nose || pip2 install nose
>>>>>> > >> > /bin/sh: pip2: command not found
>>>>>> > >> > /bin/sh: pip2: command not found
>>>>>> > >> >
>>>>>> > >> >
>>>>>> > >> > Even though we have python/2.7 installed, It looks like my
>>>>>> system has
>>>>>> > >> > python/2.6 as default. Hence, in order to install 'khmer', I
>>>>>> followed
>>>>>> > >> these
>>>>>> > >> > steps:
>>>>>> > >> >
>>>>>> > >> > module load python/2.7
>>>>>> > >> >
>>>>>> > >> > python setup.py install --user
>>>>>> > >> >
>>>>>> > >> > Apparently, it worked to install it locally, however I'm not
>>>>>> sure about
>>>>>> > >> any
>>>>>> > >> > additional file or module.
>>>>>> > >>
>>>>>> > >> OK, I would suggest running the tests inside of virtualenv.  try:
>>>>>> > >>
>>>>>> > >> curl -O
>>>>>> > >>
>>>>>> https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.x.y.tar.gz
>>>>>> > >> tar xzf virtualenv*
>>>>>> > >> cd virtualenv-*; python2.7 virtualenv.py ../khmerEnv; cd ..
>>>>>> > >> source khmerEnv/bin/activate
>>>>>> > >>
>>>>>> > >> and then 'make test'.
>>>>>> > >>
>>>>>> > >> thanks!
>>>>>> > >> --titus
>>>>>> > >>
>>>>>> > >>
>>>>>> > >
>>>>>>
>>>>>> --
>>>>>> C. Titus Brown, ctb at msu.edu
>>>>>>
>>>>>> _______________________________________________
>>>>>> khmer mailing list
>>>>>> khmer at lists.idyll.org
>>>>>> http://lists.idyll.org/listinfo/khmer
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Michael R. Crusoe:  Programmer & Bioinformatician   mcrusoe at msu.edu
>>>>>  @ the Genomics, Evolution, and Development lab; Michigan State U
>>>>> http://ged.msu.edu/ http://orcid.org/0000-0002-2961-9670 @biocrusoe
>>>>> <http://twitter.com/biocrusoe>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Michael R. Crusoe:  Programmer & Bioinformatician   mcrusoe at msu.edu
>>>  @ the Genomics, Evolution, and Development lab; Michigan State U
>>> http://ged.msu.edu/ http://orcid.org/0000-0002-2961-9670 @biocrusoe
>>> <http://twitter.com/biocrusoe>
>>>
>>
>>
>
>
> --
> Michael R. Crusoe:  Programmer & Bioinformatician   mcrusoe at msu.edu
>  @ the Genomics, Evolution, and Development lab; Michigan State U
> http://ged.msu.edu/ http://orcid.org/0000-0002-2961-9670 @biocrusoe
> <http://twitter.com/biocrusoe>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/khmer/attachments/20140707/968a1df6/attachment.htm>

More information about the khmer mailing list