[TIP] Tox command line substitutions

[Kumar McMillan]

On Fri, Jan 7, 2011 at 7:27 PM, Chris Rose <offline at offby1.net> wrote:
> Actually in a CI system is exactly when I've used the positional
> arguments; to add nose plugin configuration that enabled coverage and
> xunit output :)
>
> This is turning out to be really, really tricky; the mini language for
> substitution is not regular at all -- having to handle nested {}
> constructs is giving me the twitch.
>
> basically, I'm stumped on how, exactly, to do this, without writing my
> own lexer. I'm not even sure how to express the grammar.

Oh, I see.  As a workaround you could make a bash script called
run_tests.sh that did all the find ... -exec {} stuff and the
nosetests regex stuff.  The bash script could use $@ to capture
additional arguments and thus should work fine in tox as:

[testenv]
...
commands = run_tests.sh []

>
> On Fri, Jan 7, 2011 at 11:30 AM, Kumar McMillan
> <kumar.mcmillan at gmail.com> wrote:
>> On Fri, Jan 7, 2011 at 8:41 AM, Chris Rose <offline at offby1.net> wrote:
>>> I'd like to propose one further amendment: Change from {} as a
>>> substitution marker to {{}} (a al django templates, for example)
>>
>> maybe it would help to introduce another optional ini parameter?
>>
>> [testenv]
>> raw_commands =
>>    find ... -exec ... {}
>>    run_tests.sh
>>
>> where raw_commands cannot be extended with additional command line
>> parameters.  I think most people are using tox for CI so this might be
>> a useful alternative when needed.
>>
>>> because I can think of at least one command pattern that's pretty
>>> common (find ... -exec ... {} \;) that will trigger this.
>>>
>>> I'd call that a secondary concern, though. Getting the positional
>>> argument syntax to be correct would be more important.
>>>
>>> On Fri, Jan 7, 2011 at 7:32 AM, holger krekel <holger at merlinux.eu> wrote:
>>>> Hi Chris,
>>>>
>>>> just back from vacation - you also filed issue15 IISIC.
>>>> (And i'd like to cross-post this mail to the issue but ASFAIK unfortunately
>>>> GCode does not allow updating an issue by mail ...)
>>>>
>>>> http://code.google.com/p/pytox/issues/detail?id=15
>>>>
>>>> More inline commenting ...
>>>>
>>>> On Wed, Jan 05, 2011 at 08:48 -0700, Chris Rose wrote:
>>>>> I ran into an issue with tox command positional argument substitution
>>>>> that I was going to look into fixing, but it turns out that at least
>>>>> to some extent the issue is by design (at least, according to the unit
>>>>> tests in tox).
>>>>>
>>>>> Here's my own example, with expected behaviour:
>>>>>
>>>>> command = nosetests -m '(?:^|[\\b_\\./-])(?:unit)?[Tt]est'
>>>>>
>>>>> Tox substitutes this as:
>>>>>
>>>>> nosetests -m '(?:^|POSARGS)(?:unit)?POSARGSest'
>>>>>
>>>>> Obviously, this is undesirable.
>>>>>
>>>>> >From the unit tests, I see that by design tox is expected to support
>>>>> replacing multi-word elements in the command as denoted by []
>>>>> characters with the positional arguments, if provided, and if not, to
>>>>> strip them out completely:
>>>>>
>>>>>            [section]
>>>>>            key2=
>>>>>                cmd1 []
>>>>>                cmd2 [{item2} \
>>>>>                     other]
>>>>>
>>>>> Here, cmd2 will either be "cmd2 POSARGS", "cmd2", or "cmd2 item2value"
>>>>> depending on substitutions
>>>>>
>>>>> So, the question is, then, is this a good design for command
>>>>> substitution? \[.*\] is a VERY greedy way to replace text in a
>>>>> command, especially given that there's no way to escape these strings
>>>>> as it stands. Also, any escaping scheme is going to be subject to some
>>>>> potentially awful nested escaping issues, depending on how the command
>>>>> is executed. That might not be a problem -- I suspect that the command
>>>>> is executed using subprocess, so shell escaping is moot -- but it
>>>>> might be a hard problem to solve.
>>>>>
>>>>> For my part, I'd suggest changing the substitution marker to something
>>>>> less likely to ever be found in a command line; '{{}}' or '<<<>>>' or
>>>>> something like it, or to add a way to escape them.
>>>>
>>>> To keep things mostly compatible i like your suggestion to
>>>> (continue to) recognize "[]" literally and disallow specifying
>>>> default arguments.  We should this via a a {*} substitution
>>>> and phase out "[]" usage in the documentation and examples.
>>>> This way we end upw ith only one substitution mechanism
>>>> which makes it easier to think about escaping/providing
>>>> default values etc.
>>>>
>>>>> I'd like to contribute a fix to this, but first I need to know what
>>>>> the right fix should be. Thoughts?
>>>>
>>>> if the above makes sense i'd be happy to review and merge your fix.
>>>>
>>>> cheers,
>>>> holger
>>>>
>>>>
>>>>> --
>>>>> Chris R.
>>>>> ======
>>>>> Not to be taken literally, internally, or seriously.
>>>>> Twitter: http://twitter.com/offby1
>>>>>
>>>>> _______________________________________________
>>>>> testing-in-python mailing list
>>>>> testing-in-python at lists.idyll.org
>>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>>>
>>>>
>>>> --
>>>>
>>>
>>>
>>>
>>> --
>>> Chris R.
>>> ======
>>> Not to be taken literally, internally, or seriously.
>>> Twitter: http://twitter.com/offby1
>>>
>>> _______________________________________________
>>> testing-in-python mailing list
>>> testing-in-python at lists.idyll.org
>>> http://lists.idyll.org/listinfo/testing-in-python
>>>
>>
>
>
>
> --
> Chris R.
> ======
> Not to be taken literally, internally, or seriously.
> Twitter: http://twitter.com/offby1
>