[TIP] setup pytest fixtures in parallel

[Bruno Oliveira]

On Fri, Nov 24, 2017 at 8:34 AM Wouter Overmeire <lodagro at gmail.com> wrote:

> Thanks for the feedback. I was unfamiliar with concurrent module (Except
> for this project i'm stuck on py2).
>
> After reading the python docs and using your plan it was straight forward
> to go from a fully setup fixture to a future and setup fixtures in //.
>

Great to hear. Please note there's a backport of concurrent.futures for
Python 2.7 named `futures`: https://pypi.org/project/futures/

Cheers,
Bruno;


> Op 24 nov. 2017 10:19 schreef "Bruno Oliveira" <nicoddemus at gmail.com>:
>
> Hi Wouter,
>>
>> As Ronny said, it is currently impossible for pytest to do this for you
>> automatically, but I believe it is possible to accomplish that with some
>> work yourself:
>>
>> 1. Create a session-scoped fixture which returns a
>> concurrent.futures.ThreadPoolExecutor.
>> 2. Modify each fixture which you want to run in parallel to receive the
>> executor, and instead of doing its work directly into the body of the
>> fixture, move that to a function and post the function to the executor.
>> Return the Future object.
>> 3. Each function which uses one of the fixtures in parallel will have to
>> be modified to deal with a Future object.
>>
>> Hope that helps,
>> Bruno
>>
>> On Fri, Nov 24, 2017 at 5:32 AM RonnyPfannschmidt <
>> opensource at ronnypfannschmidt.de> wrote:
>>
>>> Hi Wouter,
>>>
>>>
>>> currently there is no way to do fixture setup in parallel and also no
>>> underlying structure to implement concurrent and/or parallel fixture
>>> creation.
>>> enabling something like it is no easy feat and requires a major rework
>>> of the internals of the pytest fixture system.
>>>
>>>
>>> -- Ronny
>>>
>>>
>>>
>>> Am 24.11.2017 um 08:02 schrieb Wouter Overmeire:
>>>
>>> Hi,
>>>
>>> Is there a way in pytest (or a plugin) to setup test fixtures in
>>> parallel?
>>> Not running the tests in // but only setup the fixtures in //.
>>>
>>> Suppose I have a number of tests and each have a different number of
>>> fixtures.
>>> The fixtures take long to setup, given they are independent of each
>>> other and not cpu limited, setup could be done in //.
>>>
>>> I prefer not to stuff all fixtures in one major fixture and control from
>>> this one how to build.
>>> Not all tests use the same fixtures, and fixtures come and go. Ideally
>>> I`m looking for a way to add additional decorator (or arg on existing
>>> fixture pytest decorator) on a fixture indicating it can be setup in //.
>>>
>>> I`m familiar with the concept of scope on fixtures, but even for session
>>> scoped fixtures I would like to setup them in //.
>>>
>>> Probably not so trivial given fixtures can be hierarchical.
>>>
>>> The hook pytest_fixture_setup looks the way to go, I`m wondering how to
>>> run rest of the fixture setup chain in a thread and return the fixture when
>>> done, This feels like breaking the api model behind hooks, given return
>>> value of hook determines to stop the chain. Not modify rest of the call
>>> chain.
>>>
>>> Wouter
>>>
>>>
>>> _______________________________________________
>>> testing-in-python mailing listtesting-in-python at lists.idyll.orghttp://lists.idyll.org/listinfo/testing-in-python
>>>
>>>
>>> _______________________________________________
>>> testing-in-python mailing list
>>> testing-in-python at lists.idyll.org
>>> http://lists.idyll.org/listinfo/testing-in-python
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20171124/e7e95503/attachment.htm>