[TIP] Modifying fixture parametrization from the command line

[Laurent Brack]

On 10/29/14, 10:29 PM, holger krekel wrote:
> On Wed, Oct 29, 2014 at 12:33 -0700, Laurent Brack wrote:
>> Hi Holger,
>>
>> On 10/29/14, 10:31 AM, holger krekel wrote:
>>> Hi Laurent,
>>>
>>> On Wed, Oct 29, 2014 at 09:22 -0700, Laurent Brack wrote:
>>>> Hi,
>>>>
>>>> I have been struggling with this issue and couldn't find a way to
>>>> make it work.
>>>>
>>>> For sake of simplicity, say I take in the example from the documentation
>>>>
>>>> http://pytest.org/latest/fixture.html#parametrizing-a-fixture
>>>>
>>>> but want to give the user the ability, to change the list of mail
>>>> servers from the command line (say smtp.gmail.com).
>>>
>>> You can't do this through direct fixture parametrization.  But there
>>> is a hook you can easily use, pytest_generate_tests.  Here is a full
>>> example probably fitting quite exactly your use case:
>>>
>>> http://pytest.org/latest/parametrize.html#basic-pytest-generate-tests-example
>>
>> Thanks. I use this hook for other things and when I saw the email
>> from Wouter this morning I was thinking of doing this, however ...
>>
>> The advantage of using fixtures is that if say reaching
>> smpt.gmail.com fails, the tests depending on that fixture would be
>> marked as error which is what you want (in our case, we use devices)
>> but I am not sure what I can do in pytest_generate_tests. Throwing
>> an exception is not want you want to do off course, and ignoring the
>> faulty device would lead to the test not showing up at all in the
>> report (makes sense?).
>
> Does "deferrring the setup" help here?
> http://pytest.org/latest/example/parametrize.html#deferring-the-setup-of-parametrized-resources
>
> If you use "indirect=True", fixture functions will still be called
> and you can fail/skip/xfail them as needed.
>
I thought I read the pytest documentation (in fact I did see this but 
didn't make the connection as I had forgotten). This is in fact exactly 
what I need. Thanks for pointing this out.
>>>
>>> Pity there is no searchable pytest cookbook yet so you could have found
>>> this more easily.
>> Here is an idea for a book :)
>
> Or a site :) I'd like to enter "parametrize command line" and end up
> with the example i originally posted and or related examples.

Yes, a site is a great idea. We have elected pytest as the defacto test 
framework because we could address so many use cases we have and so far, 
it has delivered (we actually use it to test products as you already 
know). We are even in the process of writing a plugin to trigger builds 
prior to starting test (this is a long story).

I remember the blog post you made on embedded device testing with pytest 
(http://holgerkrekel.net/2013/11/12/running-tests-against-multiple-devicesresources-in-parallel/).

we have recipes of our own which perhaps could help others. Perhaps 
subject to a review process.

This would make pytest even more attractive to perhaps a wider audience. 
Often I see people looking at pytest as a test framework for python 
only, where in fact it can be applied to a lot of things.

I would be willing to contribute. Let me know when you want to get 
started :).
>
> best,
> holger
>
>
>>>
>>> HTH,
>> It does, it confirms this can't be done via fixtures ... at least
>> not in the conventional way :)
>>> holger
>>>
>> best/Laurent
>>>
>>>> Basically
>>>>
>>>> @pytest.fixture(scope="module",
>>>>                  params=["merlinux.eu", "mail.python.org"])
>>>> def smtp(request):
>>>>      smtp = smtplib.SMTP(request.param)
>>>>      ...
>>>>
>>>> becomes
>>>>
>>>> @pytest.fixture(scope="module",
>>>>                  params=["smtp.gmail.com"])
>>>> def smtp(request):
>>>>     ...
>>>>
>>>> At runtime.
>>>>
>>>> The problem I had is that list given in "params" is created when
>>>> conftest is loaded which is before the pytest_addoption and
>>>> pytest_configure are invoked.
>>>>
>>>> I tried to use a global variable (yuk) and modify the list content
>>>> in the hook hoping that the fixture keeps the same reference but
>>>> this didn't work.
>>>>
>>>> Is there anyway I can do this?
>>>>
>>>> Thanks in advance.
>>>>
>>>> /Laurent
>>>>
>>>> _______________________________________________
>>>> testing-in-python mailing list
>>>> testing-in-python at lists.idyll.org
>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>>
>>