[TIP] Math question - related to testing, not python-specific

[Marc Tardif]

There was a mistake in my simplification which should ultimately read

  t >= ln(1 - q) / ln(1 - p)

For example, if p = .5 and you want q = .99

  t >= ln(1 - .99) / ln(1 - .5)
  t >= 7

In words, if your test fails half the time then you must run it at least
7 times if you want at least 99% confidence the failure will not get missed.

* Marc Tardif <marc at interunion.ca> [2020-02-20 16:44 -0500]:
> Assuming the attempts are independent, the number of errors E out of
> t trials follows a binomial distribution
> 
>   E ~ Bin(t, p)
> 
> The probability of an error occurring at least once is
> 
>   P(E >= 1) >= q
> 
> which is equivalent to
> 
>   1 - P(E = 0) >= q
> 
> which simplifies to
> 
>   1 - nC0 * p^0 * (1 - p)^(t - 0) >= q
>   (1 - p)^t <= 1 - q
>   t * ln(1 - p) <= ln(1 - q)
>   t <= ln(1 - q) / ln(1 - p)
>   t >= ln(1 - p) / ln(1 - q)
> 
> For example, if p = .5 and you want q = .01
> 
>   t >= ln(1 - .5) / ln(1 - .01)
>   t >= 69
> 
> * Dan Stromberg <dstromberglists at gmail.com> [2020-02-20 12:40 -0800]:
> > If a transient error fails with probability p for one invocation of a test,
> > how many times t do you have to run the test to have probability q of an
> > undetected, continued failure getting missed?
> > 
> > By a "transient error", I mean a test that fails p*100 percent of the time
> > with the same inputs - usually caused by a race condition.
> > 
> > I think I remember enough discrete math to formulate the question, but I
> > don't remember enough to answer it for myself :-S

-- 
Marc Tardif <marc at interunion.ca>
Freenode: cr3, Jabber: cr3 at jabber.org
1024D/72679CAD 09A9 D871 F7C4 A18F AC08 674D 2B73 740C 7267 9CAD