[TIP] Tox command line substitutions

[Chris Rose]

Actually in a CI system is exactly when I've used the positional
arguments; to add nose plugin configuration that enabled coverage and
xunit output :)

This is turning out to be really, really tricky; the mini language for
substitution is not regular at all -- having to handle nested {}
constructs is giving me the twitch.

basically, I'm stumped on how, exactly, to do this, without writing my
own lexer. I'm not even sure how to express the grammar.

On Fri, Jan 7, 2011 at 11:30 AM, Kumar McMillan
<kumar.mcmillan at gmail.com> wrote:
> On Fri, Jan 7, 2011 at 8:41 AM, Chris Rose <offline at offby1.net> wrote:
>> I'd like to propose one further amendment: Change from {} as a
>> substitution marker to {{}} (a al django templates, for example)
>
> maybe it would help to introduce another optional ini parameter?
>
> [testenv]
> raw_commands =
>    find ... -exec ... {}
>    run_tests.sh
>
> where raw_commands cannot be extended with additional command line
> parameters.  I think most people are using tox for CI so this might be
> a useful alternative when needed.
>
>> because I can think of at least one command pattern that's pretty
>> common (find ... -exec ... {} \;) that will trigger this.
>>
>> I'd call that a secondary concern, though. Getting the positional
>> argument syntax to be correct would be more important.
>>
>> On Fri, Jan 7, 2011 at 7:32 AM, holger krekel <holger at merlinux.eu> wrote:
>>> Hi Chris,
>>>
>>> just back from vacation - you also filed issue15 IISIC.
>>> (And i'd like to cross-post this mail to the issue but ASFAIK unfortunately
>>> GCode does not allow updating an issue by mail ...)
>>>
>>> http://code.google.com/p/pytox/issues/detail?id=15
>>>
>>> More inline commenting ...
>>>
>>> On Wed, Jan 05, 2011 at 08:48 -0700, Chris Rose wrote:
>>>> I ran into an issue with tox command positional argument substitution
>>>> that I was going to look into fixing, but it turns out that at least
>>>> to some extent the issue is by design (at least, according to the unit
>>>> tests in tox).
>>>>
>>>> Here's my own example, with expected behaviour:
>>>>
>>>> command = nosetests -m '(?:^|[\\b_\\./-])(?:unit)?[Tt]est'
>>>>
>>>> Tox substitutes this as:
>>>>
>>>> nosetests -m '(?:^|POSARGS)(?:unit)?POSARGSest'
>>>>
>>>> Obviously, this is undesirable.
>>>>
>>>> >From the unit tests, I see that by design tox is expected to support
>>>> replacing multi-word elements in the command as denoted by []
>>>> characters with the positional arguments, if provided, and if not, to
>>>> strip them out completely:
>>>>
>>>>            [section]
>>>>            key2=
>>>>                cmd1 []
>>>>                cmd2 [{item2} \
>>>>                     other]
>>>>
>>>> Here, cmd2 will either be "cmd2 POSARGS", "cmd2", or "cmd2 item2value"
>>>> depending on substitutions
>>>>
>>>> So, the question is, then, is this a good design for command
>>>> substitution? \[.*\] is a VERY greedy way to replace text in a
>>>> command, especially given that there's no way to escape these strings
>>>> as it stands. Also, any escaping scheme is going to be subject to some
>>>> potentially awful nested escaping issues, depending on how the command
>>>> is executed. That might not be a problem -- I suspect that the command
>>>> is executed using subprocess, so shell escaping is moot -- but it
>>>> might be a hard problem to solve.
>>>>
>>>> For my part, I'd suggest changing the substitution marker to something
>>>> less likely to ever be found in a command line; '{{}}' or '<<<>>>' or
>>>> something like it, or to add a way to escape them.
>>>
>>> To keep things mostly compatible i like your suggestion to
>>> (continue to) recognize "[]" literally and disallow specifying
>>> default arguments.  We should this via a a {*} substitution
>>> and phase out "[]" usage in the documentation and examples.
>>> This way we end upw ith only one substitution mechanism
>>> which makes it easier to think about escaping/providing
>>> default values etc.
>>>
>>>> I'd like to contribute a fix to this, but first I need to know what
>>>> the right fix should be. Thoughts?
>>>
>>> if the above makes sense i'd be happy to review and merge your fix.
>>>
>>> cheers,
>>> holger
>>>
>>>
>>>> --
>>>> Chris R.
>>>> ======
>>>> Not to be taken literally, internally, or seriously.
>>>> Twitter: http://twitter.com/offby1
>>>>
>>>> _______________________________________________
>>>> testing-in-python mailing list
>>>> testing-in-python at lists.idyll.org
>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>>
>>>
>>> --
>>>
>>
>>
>>
>> --
>> Chris R.
>> ======
>> Not to be taken literally, internally, or seriously.
>> Twitter: http://twitter.com/offby1
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>>
>



-- 
Chris R.
======
Not to be taken literally, internally, or seriously.
Twitter: http://twitter.com/offby1