[khmer] How to speed up the filter-below-abund script ?

[Alexis Groppi]

Hi,

Metagenome assembly :
My data :
- original (quality filtered) data : 4463243 reads (75 nt) (Illumina)
1/ Single pass digital normalization with normalize-by-median (C=20)
==> file .keep of 2560557 reads
2/ generated a hash table by load-into-counting on the .keep file
==> file .kh of ~16Go (huge file ?!)
3/ filter-below-abund with C=100 from the two previous file (table.kh 
and reads.keep)
Still running after 24 hours  :(

Any advice to speed up this step ? ... and the others (partitionning ...) ?

I can have an access to a HPC : ~3000 cores.

Thanks for your help

Alexis

-- 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/khmer/attachments/20130312/81e08323/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Signature_Mail_A_Groppi.png
Type: image/png
Size: 29033 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/khmer/attachments/20130312/81e08323/attachment-0002.png>