[TIP] Why do we need loadTestsFromPackage ?

[Olemis Lang]

On Wed, Apr 15, 2009 at 3:55 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Olemis Lang wrote:
>> On Tue, Apr 14, 2009 at 12:11 PM, Michael Foord
>> <fuzzyman at voidspace.org.uk> wrote:
>>> Olemis Lang wrote:
>>>
>>>>> what would the interface to the
>>>>> discovery mechanism look like?
>>>>
>>>> {{{
>>>> #!python
[...]
>>>> }}}
>>>
>>> Shoehorning discovery into loadTestsFromModule is worse - it is less
>>> amenable to customization as you can't customize loading tests from a
>>> module
>>> whilst still reusing the discovery mechanism.
>>
>> I'll try to show you the way to refine a loader in the approach I
>> mentioned.
[...]
>
> Except that with a module that is also a package namespace, delegating to
> loadTestsFromModule will invoke discovery.
>

Not exactly. On employing a simple loader (e.g. Loader1 in the
previous code snippets ) then only the tests in the top-level module
are loaded.

{{{
#!python
simpleloader = Loader1()
s1 = simpleloader.loadTestsFromModule(pkg)
}}}

On employing a wrapper loader (e.g. DiscoLoader in the previous code
snippets ) then all the tests across the package hierarchy are loaded.

{{{
#!python
simpleloader = Loader1()
discoloader = DiscoLoader(simpleloader)
s2 = discoloader.loadTestsFromModule(pkg)
}}}

> If discovery is in a separate method which calls loadTestsFromModule then
> you can overload module loading separately from the discovery mechanism.

Excuse me, but here I c your point, but still I strongly disagree with
loadTestsFromPackage. All this can be done in the approach I mention,
without introducing loadTestsFromPackage, but a wrapper loader.

> It
> would be a mistake to conflate them.
>

«It would be a mistake to conflate them» is not a concrete
argument and as far as I can tell doesn't actually say anything.

I still think that the creating a new method is not needed.

>>> Whether it is provided on the standard loader or on a subclass is not a
>>> very
>>> interesting question.
>>
>> Please, just to be sure. The «it» above in «Whether it is provided on
>> » refers to what ?
>
> It being discovery.

I dont think following this will be useful

>>>
>>> "is more dynamic and is suitable for diverse discovery strategies,
>>> varying
>>> apart of the code for the real | concrete loader." is not a concrete
>>> argument and as far as I can tell doesn't actually say anything.
>>>
>>
>> Provided that DiscoLoader holds a reference to the real loader to use,
>> by more dynamic I mean
>>
>> {{{
>> l1 = Loader1()
>> l2 = Loader2()
>> discoloader = DiscoLoader(l1)
>> s1 = discoloader.loadTestsFromModule(pkg)   # Blue Tests
>>
>>     # in whole package
>> discoloader.loader = l2
>> s2 = discoloader.loadTestsFromModule(pkg)   # Dark blue Tests
>>
>>     # in whole package
>>
>> # And the instance of DiscoLoader is the same
>
> You're still using two different loader instances (three in point of fact).
> How is the above better than:
>
> loader1 = Loader1()
> loader2 = Loader2()
>
> s1 = loader1.loadTestsFromPackage(pkg)   # Blue Tests
> s2 = loader2.loadTestsFromPackage(pkg)   # Dark Blue Tests
>

Could you sketch a little the implementation of these classes, and
specially how will they change in case I want to use two different
discovery strategies ? In my case it works as follows :

{{{
#! python

# Consider Loader1 and Loader2 are already there ;)

class DiscoLoader3(DiscoLoader):
   def __init__(self, loader):
      self.loader = loader
      # more
   def loadTestsFromModule(self, module):
      suite = self.suiteClass()
      for x in disco_strategy1(module):
          suite.addTest(self.loader.loadTestsFromModule(x))
      return suite

class DiscoLoader4(DiscoLoader):
   def __init__(self, loader):
      self.loader = loader
      # more
   def loadTestsFromModule(self, module):
      suite = self.suiteClass()
      for x in disco_strategy20001(module):
          if self.myfilter(x):
            suite.addTest(self.loader.loadTestsFromModule(x))
          elif self.runtime_condition(*args):
            suite.addTest(self.loader.loadTestsFromModule(x))
      return suite

l1 = Loader1()
l2 = Loader2()
discoloader = DiscoLoader(l1)
discoloader1 = DiscoLoader1(l1)
discoloader3 = DiscoLoader3(l1)
discoloader4 = DiscoLoader4(l1)

all_blue_suites = [x.loadTestsFromModule(x) for x in (l1, discoloader,
discoloader1, discoloader3, discoloader4)]

discoloader.loader = l2
discoloader1.loader = l2
discoloader3.loader = l2
discoloader4.loader = l2

all_dark_blue_suites = [x.loadTestsFromModule(x) for x in (l2,
discoloader, discoloader1, discoloader3, discoloader4)]

}}}

If you realize I cover here the following cases (considering all the
snippets written in the thread ;):

|| Wrapper || Loader1 || Loader2 ||
|| No wrapper || only blue tests  || only dark blue tests  ||
|| DiscoLoader || blue tests with initial disco || dark blue tests
with initial disco ||
|| DiscoLoader1 || blue tests with disco strategy 1|| dark blue tests
with disco strategy 1||
|| DiscoLoader3 || blue tests with disco strategy 3|| dark blue tests
with disco strategy 3||
|| DiscoLoader4 || blue tests with disco strategy 4|| dark blue tests
with disco strategy 4||

I mean, this way you implement 2 concrete loaders + 4 disco loaders
(classes) and get back 10 different suites in a uniform manner. In
general, in case of having m concrete loaders and n disco strategies,
you get back m * (n + 1) different suites in a uniform manner.

>
> If both Loader1 and Loader2 inherit from the default loader and override
> loadTestsFromModule to implement their different semantics then it is easier
> to understand and you have lost nothing in terms of dynamism.
>

I have some arguments to say so far, but unless you mention how your
proposal is gonna work I am in the position of imagining what you are
thinkin' considering what you said (not a good position, I mean it,
I'm quite bad trying to guess AYCS).

Could you please be so kind and sketch how the situations (code
snippets) I mentioned are gonna be handled according to the ideas you
have about loadTestsFromPackage ?

> In actual fact your pattern of subclassing loader, implementing different
> functionality in an overridden method but delegating to another instance is
> functionally identical to (and would be clearer expressed) as a single
> function that takes a loader instance as one of the arguments. Personally I
> find your suggested pattern bizarre and an abuse of OO.
>

I feel that so far the debate has been positive with
respect to what I was saying : Firstly,
there was no way to override the discovery strategy, now there is,
but we need three classes, plus the other arguments you mentionned ...
which I understand and respect, even if I still have other ideas.

The only things I'm gonna say is the following:

  - Python gives an uniform treatment to packages and modules (the
    class that represents both of them, as well as other things). So,
    at least IMHO, one of the ideas I have (with a strong dose of
    «purity» so far ;) is that it could be valuable to preserve such uniformity
    in test code too, and treat both just as usual, like a single
    «thing». For Python, this is valuable when the source code grows
    in a manner that a single module is not just enough (or is a bad
    idea) and people dont notice the change. For test code this could
    be cool for maintaining the test code.
  - Secondly the structure I am proposing looks quite similar to the
    widely adopted (in many languages) I/O classes and which has been
    recently included in Py3k through «PEP: 3116 New I/O». IMHO
    they are so succssful since they 'r closely related to the
    structure -full of purity & patterns- presented by GoF in that
    book (you may discard the last sentence if you want to since
    that's IMHO ;) That PEP mentions in Rationale section :

    «Python needs a specification for basic byte-based I/O streams
    to which we can add buffering and text-handling features.
    Once we have a defined raw byte-based I/O interface, we can add
    buffering and text handling layers on top of any byte-based I/O class.
    The same buffering and text handling logic can be used for files,
    sockets, byte arrays, or custom I/O classes developed by Python
    programmers.  »

    IMHO in our specific case this could be written

    «unittest needs a specification for basic style-specific test
    loaders to which we can add discovery and assembly features.
    Once we have a (defined | concise) «style-oriented» interface, we
    can add
    discovery, assembly, cloning, mutation, filtering and more
    elaborate features on top of any «style-specific» test loader.
    The same discovery, assembly, cloning, mutation, and filtering
    logic can be used for FIT(Ness), unittest TestCase, doctests
    py.test, Peckcheck, FitLoader, TextTest (... more ...), or custom
    formats developed by Python programmers in order
    to write different kinds of tests.  »

At least that's the approach followed by dutest (discard this
comment if you feel it's noisy)

In fact IMHO much of what's been said in PEP 3116 could be
rewritten for test discovery (and other features), by only varying
some nouns & verbs

In a separate message I'll provide examples of the similarities of
the approach I mention and PEP 3116 (with respect to structure), and
in another one (after I actually see the examples you could provide)
I'll talk about the comments you have sent back to the list.

Waiting from the other side to see your examples

Thnx in advance ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article: