[TIP] Coming changes to unittest2 plugins

[Olemis Lang]

On Tue, Sep 28, 2010 at 7:17 PM,  <exarkun at twistedmatrix.com> wrote:
> On 09:04 pm, olemis at gmail.com wrote:
>> On Tue, Sep 28, 2010 at 4:41 PM,  <exarkun at twistedmatrix.com> wrote:
>>> On 07:57 pm, olemis at gmail.com wrote:
>>>> On Mon, Sep 27, 2010 at 8:36 PM,  <exarkun at twistedmatrix.com> wrote:
>>>>> On 27 Sep, 07:42 pm, holger at merlinux.eu wrote:
>>>>>> On Mon, Sep 27, 2010 at 19:19 -0000, exarkun at twistedmatrix.com wrote:
>>>>>>>>
>>>>>>>> packaging meta data is for consumption of tools.  For example, i
>>>>>>>> want
>>>>>>>> to
>>>>>>>> write one that converts a newstyle setup.cfg into an oldstyle
>>>>>>>> setup.py.
>>>>>>>
>>>>>>> What transformation do you need to perform on an arbitrary,
>>>>>>> application-
>>>>>>> specified fully qualified Python name?  In general you can't know
>>>>>>> anything about it, whether it has a ":" in it or not.
>>>>>>
>>>>>> Wrong. The ":" carries information.  For example, if we have:
>>>>>>
>>>>>>   [global]
>>>>>>   script-entry-points =
>>>>>>       somescript = testpath.to:some.function
>>>>>>
>>>>>> we can easily generate an oldstyle setup.py from it:
>>>>>>
>>>>>>   setup(entry_points={'somescript': 'testpath.to:some.function'}, ...)
>>>>>>
>>>>>> without having to load live objects.  The dotted name does
>>>>>> not allow that and thus carries less information.
>>>>>
>>>>> Thanks for explaining that.  This is clearly a case where the ":" helps
>>>>> (a
>>>>> sort of self-referential case, where the ":" is only needed because it
>>>>> used
>>>>> to be needed).
>>>>>
>>>>> It's not an important case to me (and I don't think it should be
>>>>> important
>>>>> to anyone else).  It's still nice to know *why* the ":" might be
>>>>> desired
>>>>> though.
>>>>
>>>> What d'u think of the example I mentioned before (i.e. the Trac plugin
>>>> (macro?)  for the lazy dev ...) ???
>>>
>>> This one?
>>
>> kind of ...
>
> Why "kind of"?  It's a direct quote from your email.  If you meant something
> else, please say what.

pls , don't get over-excited , it's just an expression .

>>>>
>>>> However, suppose I'm developing some apps (each one under top-level
>>>> folder in VCS) adding unittest plugins and I want to develop a Trac
>>>> plugin (macro ?) that will know the path to top-level folder in VCS
>>>> and will look for unittest metadata definition inside setup.* (or
>>>> wherever it might be defined ...) of all direct descendants (i.e.
>>>> folders). With this data it will render inside a floating div some
>>>> links to Trac repository browser area (e.g. highlighting the lines
>>>> where the plugin code is located) . That would allow me to quickly
>>>> find plugin code inside e.g. repository browser and wiki pages (/me
>>>> the lazy dev ) .
>>>
>>> I have a little difficulty following it.  That said, unless I've gravely
>>> misunderstood, it sounds like you can implement it without ":" just about
>>> as
>>> easily as with.  Given "foo.bar.baz" look for foo/bar/baz.py. Exists?
>>>  Great, you're done.  Otherwise look for foo/bar.py.  Otherwise look for
>>> foo.py.
>>
>> and this backtracking is definitely quite much more inefficient than
>> loading the module directly . I hope you know that in this case I'm
>> not talking about flat folder structure but rather about data stored
>> in VCS changeset store , and streams are not file streams , but VCS
>> internal streams .
>
> No, I don't know that.  Maybe you should have a more efficient data store?

Trac already has a VCS cache, but it makes a real difference as
compared to direct FS access. In this case I could not control the
details of how VCS store and retrieve changesets, nor even the details
of how Trac connectors instantiate and offer all those objects to
plugins)

It goes far beyond *me* (supposing I'm the one who'll develop this BTW)

>  It's only approximately a constant difference, anyway.  I'm sure you spend
> more time thrashing the locks on your database or rendering html, so it
> probably doesn't matter.

Let's supose all this is true, but you are gonna introduce all those
performance penalties, ambiguities and backtracking, just because
colons are ugly ???

>>> It'll be wrong sometimes, but so will assuming that "foo.bar:baz" means
>>> to
>>> look in foo/bar.py.
>>
>> no way if I say so . If that's wrong then there's an error in my code,
>> so I better look a way to catch situations like these ...
>
> Consider the possibility that baz is implemented in foo/_baz.py and exported
> via foo/bar.py.  The proper public name of the interface is foo.bar.baz, not
> foo._baz.baz.  But there's no way you can learn that from "foo.bar:baz"

yes there is , in this case I'm talking about the macro'd highlight
`from foo._baz import baz` inside `foo/bar.py` and , if I decide to do
so , I can follow imports , but if I don't , that's it !!!

;o)

> or
> "foo.bar.baz".  So if you think you'll find the source for baz in foo/bar.py
> because you saw either of those strings, you may be in for a surprise.

No, I'm just talking in first place about getting in O(1) without
loading live Py objects the location (module + path to object ,
considering current Py import semantics) where something  (e.g.
unittest2 plugins) is declared , available , ... If it was created
somewhere else or whatever , that's secondary for the purposes of
distutils entry points .

>>>
>>> Without loading the code you can't be completely sure
>>
>> No way . I'm writing the code (and let's say I figured out to ensure
>> it will be there before committing to VCS ;o) so how could I be wrong
>> ?
>
> I don't really understand this, but hopefully my explanation above addresses
> it.

I'm just saying that I obviuosly know where it is cause I wrote the
code , the entry points and let's say, wrote a test to ensure that
objects are where they should be . Then , how could I be wrong ?

>>
>> IMO there's a difference between not to know because the syntax is
>> ambiguous and not to find it in the place it was supposed to be
>> because there's a bug . In the first case you are on your own &
>> hopeless , in the second you'd be terribly mistaken .
>
> As above.

What as above ...

>>>
>>> (and maybe even if you load the code you'll still get it wrong sometimes,
>>> I'm not sure).
>>
>> even if you are able load the code from VCS using Trac multi-VCS API
>
> Involving a VCS and trac sort of muddies the waters.  The problem has
> nothing to do with either of these things,

Somehow yes (but generally speaking NO, ok) . It's just an example of
what could be done with entry points . The Trac part is not exactly
the point but the fact is that people might want to use entry point
definition for more things , not just loading Py live objects ; and
IMO it's not a good decision to constrain users or add further
complexity just because colons are ugly .

> it has to do with the fact that
> an identifier like "foo.bar.baz" or "foo.bar:baz" doesn't tell you where the
> source code for something is,

I agree , but that's not the point . The point is that :

- If I want to load the live Py object the process is O(1)
  and a two liner BTW
- It's possible that somewhere in `foo.bar.baz` there is
  a statement illustrating how this name is bound to
  the module's namespace, if it's an external symbol
  I can follow the reference .
- If that's not the case , and I'm performing static
  analysis then I'm out of look , but I'd also be hopeless
  if using *dot notation* .
- And if I want to , I can use the *colon* notation to do
  whatever I can do using the *dot* notation.

I understand that the *dot* notation may be useful in cases where
uniform global access to objects really matters (e.g. Trac's PyDoc
plugin , load unittest2 plugins , ...) but if you want to use entry
points for something little way beyond that then the *dot* notation
does not help .

> it tells you a name which you can use to get
> an object.  That's a very strong hint about where the source is, but it's
> not a guarantee.

even if this is not the point IMO , this is correct, please read above .

>>
>> (something you might not want to do inside Trac unless we'are talking
>> of installed plugins & apps ;o) you'll be guessing quite a lot of
>> times (like I said before, this exact backtracking is the one I used
>> when I patched Trac PyDocPlugin to make it compliant with PEP 302
>> import hooks e.g. egg imports, so I know it very well ;o)
>
> I don't really understand this.  You don't need to guess if you're doing the
> import.  The import will take care to resolve everything.

you start with full path to «target object» but if imports fails , you
need to retry the import with a shorter object path, isn't it ? ...

> But several of
> the sentences in this paragraph appear to contradict each other

could you explain what you see as contradictory ...

> so I'm not really sure what point you're
> making.
>

above

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article: