<div dir="ltr">Have you seen <a href="https://github.com/eventbrite/nose-knows">https://github.com/eventbrite/nose-knows</a>? I haven't tried it or dug into the source but the approach sounds similar.</div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 16, 2015 at 3:46 PM, Robert Collins <span dir="ltr"><<a href="mailto:robertc@robertcollins.net" target="_blank">robertc@robertcollins.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 17 March 2015 at 07:43, Thomi Richards <<a href="mailto:thomi.richards@canonical.com">thomi.richards@canonical.com</a>> wrote:<br>
> Hi,<br>
><br>
> On Sat, Mar 14, 2015 at 3:57 AM, Tibor Arpas <<a href="mailto:tibor.arpas@infinit.sk">tibor.arpas@infinit.sk</a>> wrote:<br>
>><br>
>> E.g. it automatically selects only tests dependent on recent changes and<br>
>> deselects all the others. The concept is quite general, technically feasible<br>
>> and it's possible to finish this and also implement the same for other test<br>
>> runners if there is interest.<br>
><br>
><br>
><br>
> This seems.. too good to be true. Would you mind explaining a bit about how<br>
> it works? Off the top of my head, I can't think of any reliable way to<br>
> determine a list of test cases that need to be run, given a diff of source<br>
> code. Am I missing something?<br>
<br>
</div></div>There was a thread on twitter about this recently<br>
<a href="https://twitter.com/ncoghlan_dev/status/566373773830397952" target="_blank">https://twitter.com/ncoghlan_dev/status/566373773830397952</a><br>
<br>
The idea goes back years :).<br>
<br>
The basic idea is to have an oracle that tells you working tree state<br>
-> tests to run<br>
<br>
Some oracles:<br>
- naming conventions; name tests such that you can tell the modules<br>
they are relevant to.<br>
pros: easy to maintain, helps mental association to the importance of layers.<br>
cons: very easy to fail to run tests where unexpected layer<br>
associations have formed<br>
- trace based: using a debug/profiling hook build a mapping of test X<br>
ran lines Y. Whenever you run the test again<br>
update the database, and map backwards from diffs to the db. You<br>
can in principle also use this to decide what<br>
tests need to run when changing a dependency, though I'm not aware<br>
of anyone doing that yet.<br>
pros: much more reliable at determining what tests to run<br>
cons: have to build the db first, and maintain it as e.g. lines<br>
move around, which makes first-time use expension<br>
- stochastically: run some subset of tests randomly, perhaps biased<br>
by naming conventions or other data like most-recently changed. Aegis<br>
uses this to select tests to run.<br>
pros: very simple<br>
cons: reproducability, and lack of coverage<br>
<br>
There was an implementation of trace based selection put together for<br>
LP about 8 years ago, and I did a thing for bzr shortly after that -<br>
there are implementations all over the place ;). None in the Python<br>
world that are generally reusable and consumable until recently AFAIK<br>
though.<br>
<br>
-Rob<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Robert Collins <<a href="mailto:rbtcollins@hp.com">rbtcollins@hp.com</a>><br>
Distinguished Technologist<br>
HP Converged Cloud<br>
<br>
_______________________________________________<br>
testing-in-python mailing list<br>
<a href="mailto:testing-in-python@lists.idyll.org">testing-in-python@lists.idyll.org</a><br>
<a href="http://lists.idyll.org/listinfo/testing-in-python" target="_blank">http://lists.idyll.org/listinfo/testing-in-python</a><br>
</font></span></blockquote></div><br></div>