Ah, I see. It seems if every kmer of every read is checked, no error-containing reads would ever be discarded. But if only a representative set of kmers from each read is checked, then that makes much more sense. I guess I need to read up on the median kmer estimator.<br><br>Thanks,<br>Daniel<br><div class="gmail_quote">On Sat, Dec 6, 2014 at 7:20 AM C. Titus Brown <<a href="mailto:ctb@msu.edu">ctb@msu.edu</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Fri, Dec 05, 2014 at 08:53:31AM -0800, C. Titus Brown wrote:<br>
> On Fri, Dec 05, 2014 at 04:49:27PM +0000, Daniel Standage wrote:<br>
> > Greetings!<br>
> ><br>
> > I have a quick question. I understand the primary motivation behind digital<br>
> > normalization, the idea of discarding data without losing any information.<br>
> > My question is about the claim that diginorm retains all real kmers while<br>
> > discarding erroneous ones. After reading over the arXiv preprint again, it<br>
> > seems this claim is independent of the three-pass protocol which does<br>
> > additional error correction.<br>
> ><br>
> > If we assume that errors are present in low abundance, why would diginorm<br>
> > ever discard a read containing an error? Wouldn't the same error have to be<br>
> > present a certain number of times before the associated kmers had<br>
> > sufficient coverage to discard those reads? In that case, we're much less<br>
> > confident that it's not real variation. Or are there probabilistic data<br>
> > structures involved that discard likely errors?<br>
> ><br>
> > Thanks!<br>
> > Daniel<br>
><br>
> Hey Daniel,<br>
><br>
> More/better answer later, but look at the part of the paper where we talk<br>
> about losing tips of contigs in the mRNAseq simulation. The median k-mer count<br>
> cannot tell the difference between undersampled contig edges and errors (which<br>
> may occur in real data sets).<br>
><br>
> But good question :)<br>
<br>
Hah, I think I misunderstood your question the first time 'round.<br>
<br>
Erroneous k-mers are present in every read with an error, so if any reads are<br>
discarded that have errors in them, erroneous k-mers are discarded along with<br>
that read. So if you have a coverage of 100 and 80% of those reads are<br>
discarded, then roughly 80% of the errors in your original data set also go<br>
away.<br>
<br>
The trick is really that using the median k-mer estimator allows us to ask if<br>
*most* of a read is new, and so if two otherwise identical (or mostly<br>
overlapping) reads have different errors, diginorm will regard them as<br>
the same anyway.<br>
<br>
HTH!<br>
<br>
cheers,<br>
--titus<br>
</blockquote></div>