[TIP] setup pytest fixtures in parallel

Wouter Overmeire lodagro at gmail.com
Fri Nov 24 02:34:48 PST 2017


Thanks for the feedback. I was unfamiliar with concurrent module (Except
for this project i'm stuck on py2).

After reading the python docs and using your plan it was straight forward
to go from a fully setup fixture to a future and setup fixtures in //.

Op 24 nov. 2017 10:19 schreef "Bruno Oliveira" <nicoddemus at gmail.com>:

> Hi Wouter,
>
> As Ronny said, it is currently impossible for pytest to do this for you
> automatically, but I believe it is possible to accomplish that with some
> work yourself:
>
> 1. Create a session-scoped fixture which returns a concurrent.futures.
> ThreadPoolExecutor.
> 2. Modify each fixture which you want to run in parallel to receive the
> executor, and instead of doing its work directly into the body of the
> fixture, move that to a function and post the function to the executor.
> Return the Future object.
> 3. Each function which uses one of the fixtures in parallel will have to
> be modified to deal with a Future object.
>
> Hope that helps,
> Bruno
>
> On Fri, Nov 24, 2017 at 5:32 AM RonnyPfannschmidt <
> opensource at ronnypfannschmidt.de> wrote:
>
>> Hi Wouter,
>>
>>
>> currently there is no way to do fixture setup in parallel and also no
>> underlying structure to implement concurrent and/or parallel fixture
>> creation.
>> enabling something like it is no easy feat and requires a major rework of
>> the internals of the pytest fixture system.
>>
>>
>> -- Ronny
>>
>>
>>
>> Am 24.11.2017 um 08:02 schrieb Wouter Overmeire:
>>
>> Hi,
>>
>> Is there a way in pytest (or a plugin) to setup test fixtures in parallel?
>> Not running the tests in // but only setup the fixtures in //.
>>
>> Suppose I have a number of tests and each have a different number of
>> fixtures.
>> The fixtures take long to setup, given they are independent of each other
>> and not cpu limited, setup could be done in //.
>>
>> I prefer not to stuff all fixtures in one major fixture and control from
>> this one how to build.
>> Not all tests use the same fixtures, and fixtures come and go. Ideally
>> I`m looking for a way to add additional decorator (or arg on existing
>> fixture pytest decorator) on a fixture indicating it can be setup in //.
>>
>> I`m familiar with the concept of scope on fixtures, but even for session
>> scoped fixtures I would like to setup them in //.
>>
>> Probably not so trivial given fixtures can be hierarchical.
>>
>> The hook pytest_fixture_setup looks the way to go, I`m wondering how to
>> run rest of the fixture setup chain in a thread and return the fixture when
>> done, This feels like breaking the api model behind hooks, given return
>> value of hook determines to stop the chain. Not modify rest of the call
>> chain.
>>
>> Wouter
>>
>>
>> _______________________________________________
>> testing-in-python mailing listtesting-in-python at lists.idyll.orghttp://lists.idyll.org/listinfo/testing-in-python
>>
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20171124/34284f6c/attachment-0001.htm>


More information about the testing-in-python mailing list