[TIP] Math question - related to testing, not python-specific
Marc Tardif
marc at interunion.ca
Thu Feb 20 13:44:20 PST 2020
Assuming the attempts are independent, the number of errors E out of
t trials follows a binomial distribution
E ~ Bin(t, p)
The probability of an error occurring at least once is
P(E >= 1) >= q
which is equivalent to
1 - P(E = 0) >= q
which simplifies to
1 - nC0 * p^0 * (1 - p)^(t - 0) >= q
(1 - p)^t <= 1 - q
t * ln(1 - p) <= ln(1 - q)
t <= ln(1 - q) / ln(1 - p)
t >= ln(1 - p) / ln(1 - q)
For example, if p = .5 and you want q = .01
t >= ln(1 - .5) / ln(1 - .01)
t >= 69
* Dan Stromberg <dstromberglists at gmail.com> [2020-02-20 12:40 -0800]:
> If a transient error fails with probability p for one invocation of a test,
> how many times t do you have to run the test to have probability q of an
> undetected, continued failure getting missed?
>
> By a "transient error", I mean a test that fails p*100 percent of the time
> with the same inputs - usually caused by a race condition.
>
> I think I remember enough discrete math to formulate the question, but I
> don't remember enough to answer it for myself :-S
--
Marc Tardif <marc at interunion.ca>
Freenode: cr3, Jabber: cr3 at jabber.org
1024D/72679CAD 09A9 D871 F7C4 A18F AC08 674D 2B73 740C 7267 9CAD
More information about the testing-in-python
mailing list