From nicoddemus at gmail.com  Thu Aug 15 18:42:29 2019
From: nicoddemus at gmail.com (Bruno Oliveira)
Date: Thu, 15 Aug 2019 22:42:29 -0300
Subject: [TIP] pytest 5.1.0 released
Message-ID: <CA+RQFfffPGFgjtj6AiZOBgnrZM5Qp05H0ANYzBzF2mvuM7d99w@mail.gmail.com>

The pytest team is proud to announce the 5.1.0 release!

pytest is a mature Python testing tool with more than a 2000 tests
against itself, passing on many different interpreters and platforms.

This release contains a number of bugs fixes and improvements, so users are
encouraged
to take a look at the CHANGELOG:

    https://docs.pytest.org/en/latest/changelog.html

For complete documentation, please visit:

    https://docs.pytest.org/en/latest/

As usual, you can upgrade from pypi via:

    pip install -U pytest

Thanks to all who contributed to this release, among them:

* Albert Tugushev
* Alexey Zankevich
* Anthony Sottile
* Bruno Oliveira
* Daniel Hahler
* David R?thlisberger
* Florian Bruhin
* Ilya Stepin
* Jon Dufresne
* Kaiqi
* Max R
* Miro Hron?ok
* Oliver Bestwalter
* Ran Benita
* Ronny Pfannschmidt
* Samuel Searles-Bryant
* Semen Zhydenko
* Steffen Schroeder
* Thomas Grainger
* Tim Hoffmann
* William Woodall
* Wojtek Erbetowski
* Xixi Zhao
* Yash Todi
* boris
* dmitry.dygalo
* helloocc
* martbln
* mei-li


Happy testing,
The Pytest Development Team
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190815/9f5db2d4/attachment.htm>

From drsalists at gmail.com  Tue Aug 27 12:46:21 2019
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 27 Aug 2019 12:46:21 -0700
Subject: [TIP] runTest?
Message-ID: <CAGGBd_raSLi84NEO0MNiMJFxG+sHk6YCcb195xTwtrgBBvb4iA@mail.gmail.com>

Hi folks.

I'm working on a large Python 3/Python 2 codebase that has several
automated test cases and test classes.  I know Python 2 is going away soon.

We have lots of test* methods in classes that inherit
from unittest.TestCase. Each of those classes also provides a runTest
method.

It's bugging me that the runTest method is getting used, and the test*
methods are getting ignored, unless we call the test* methods from the
runTest method.

In fact, if I rename the runTest method from a test class to something
unused, I get a traceback:
runTest (tests.test_disposable_locks.TestDisposableLocks)
No test ... Traceback (most recent call last):
  File "test_runner.py", line 98, in <module>
    sys.exit(main())
  File "test_runner.py", line 90, in main
    result = runner.run(test_suite)
  File "/usr/lib/python3.6/unittest/runner.py", line 176, in run
    test(result)
  File "/usr/lib/python3.6/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.6/unittest/suite.py", line 122, in run
    test(result)
  File "/usr/lib/python3.6/unittest/case.py", line 653, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.6/unittest/case.py", line 580, in run
    testMethod = getattr(self, self._testMethodName)
AttributeError: 'TestDisposableLocks' object has no attribute 'runTest'

I've been pulling my hair out trying to figure out why unittest is
preferring runTest over test*.  Isn't unittest supposed to fall back on
runTest only if there are no test* methods?

Or do I need something like:
suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
(from https://docs.python.org/2/library/unittest.html)
?  I tried getting this going with pytest a while back, but eventually set
the project aside a while - maybe that was where I got the idea that
runTest should be ignored if test* methods are present....

I still hope to use pytest with this code at some point, but for now I'd be
happy if the unittest test discovery just worked the way I expect.  My main
goals for switching to pytest is to parallelise the tests, and to stop
having to list "important" tests in a runTest method lest they be ignored.

Also, I'm finding that sometimes raising an exception terminates the
testing for the entire suite, rather than leading to a failed/errored test
and continuing, presumably because "runTest" is "the test", so if it errors
out anywhere, that's it for "that test".  If pytest can help with that,
that'd be awesome.

What can I do to get runTest out of the mix when test* methods are present?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190827/01f029b6/attachment.htm>

From chris.jerdonek at gmail.com  Tue Aug 27 13:33:48 2019
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Tue, 27 Aug 2019 13:33:48 -0700
Subject: [TIP] runTest?
In-Reply-To: <CAGGBd_raSLi84NEO0MNiMJFxG+sHk6YCcb195xTwtrgBBvb4iA@mail.gmail.com>
References: <CAGGBd_raSLi84NEO0MNiMJFxG+sHk6YCcb195xTwtrgBBvb4iA@mail.gmail.com>
Message-ID: <CAOTb1wf-r6k03OaqCCEGKZ55TeU2baBkT9+g8p5Dj2bnymTYHA@mail.gmail.com>

Why does each of your classes have a runTest method in the first place?
What is its purpose ?- what does it do?

?Chris

On Tue, Aug 27, 2019 at 12:51 PM Dan Stromberg <drsalists at gmail.com> wrote:

>
> Hi folks.
>
> I'm working on a large Python 3/Python 2 codebase that has several
> automated test cases and test classes.  I know Python 2 is going away soon.
>
> We have lots of test* methods in classes that inherit
> from unittest.TestCase. Each of those classes also provides a runTest
> method.
>
> It's bugging me that the runTest method is getting used, and the test*
> methods are getting ignored, unless we call the test* methods from the
> runTest method.
>
> In fact, if I rename the runTest method from a test class to something
> unused, I get a traceback:
> runTest (tests.test_disposable_locks.TestDisposableLocks)
> No test ... Traceback (most recent call last):
>   File "test_runner.py", line 98, in <module>
>     sys.exit(main())
>   File "test_runner.py", line 90, in main
>     result = runner.run(test_suite)
>   File "/usr/lib/python3.6/unittest/runner.py", line 176, in run
>     test(result)
>   File "/usr/lib/python3.6/unittest/suite.py", line 84, in __call__
>     return self.run(*args, **kwds)
>   File "/usr/lib/python3.6/unittest/suite.py", line 122, in run
>     test(result)
>   File "/usr/lib/python3.6/unittest/case.py", line 653, in __call__
>     return self.run(*args, **kwds)
>   File "/usr/lib/python3.6/unittest/case.py", line 580, in run
>     testMethod = getattr(self, self._testMethodName)
> AttributeError: 'TestDisposableLocks' object has no attribute 'runTest'
>
> I've been pulling my hair out trying to figure out why unittest is
> preferring runTest over test*.  Isn't unittest supposed to fall back on
> runTest only if there are no test* methods?
>
> Or do I need something like:
> suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
> (from https://docs.python.org/2/library/unittest.html)
> ?  I tried getting this going with pytest a while back, but eventually set
> the project aside a while - maybe that was where I got the idea that
> runTest should be ignored if test* methods are present....
>
> I still hope to use pytest with this code at some point, but for now I'd
> be happy if the unittest test discovery just worked the way I expect.  My
> main goals for switching to pytest is to parallelise the tests, and to stop
> having to list "important" tests in a runTest method lest they be ignored.
>
> Also, I'm finding that sometimes raising an exception terminates the
> testing for the entire suite, rather than leading to a failed/errored test
> and continuing, presumably because "runTest" is "the test", so if it errors
> out anywhere, that's it for "that test".  If pytest can help with that,
> that'd be awesome.
>
> What can I do to get runTest out of the mix when test* methods are present?
>
> Thanks!
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190827/f0282957/attachment.htm>

From chris.jerdonek at gmail.com  Tue Aug 27 16:31:01 2019
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Tue, 27 Aug 2019 16:31:01 -0700
Subject: [TIP] runTest?
In-Reply-To: <CAGGBd_pxsXVKL-Bd2d_c2niWgC6Yp9CbRa7DurFtfO5UgyimqQ@mail.gmail.com>
References: <CAGGBd_raSLi84NEO0MNiMJFxG+sHk6YCcb195xTwtrgBBvb4iA@mail.gmail.com>
	<CAOTb1wf-r6k03OaqCCEGKZ55TeU2baBkT9+g8p5Dj2bnymTYHA@mail.gmail.com>
	<CAGGBd_pxsXVKL-Bd2d_c2niWgC6Yp9CbRa7DurFtfO5UgyimqQ@mail.gmail.com>
Message-ID: <CAOTb1wcQ3RXLco9KuAUZA+aE5JYHsUGY6HEa-OPBhgEMuh65rQ@mail.gmail.com>

In my experience, I've never needed to add a runTest() method to TestCase
classes, and if a TestCase class lacks any test methods, it will simply run
no tests -- not try to run "runTest." If you look at unittest's source
code, you'll see that "runTest" is the default test name if no method name
is passed to TestCase's constructor. I'm guessing the behavior you're
experiencing is due to a custom test runner or discovery or some other
customization under your control. I've never seen this behavior from
out-of-the-box uses of unittest.

--Chris


On Tue, Aug 27, 2019 at 1:40 PM Dan Stromberg <drsalists at gmail.com> wrote:

>
> It calls each "important" test in turn.  It's like a poor-person's
> discovery. If a test is left out of runTest by accident, it gets ignored.
>
> Like:
>     def runTest(self):
>         assert self.test_red()
>         assert self.test_green()
>         assert self.test_blue()
>
> Or:
>         def runTest(self):
>             self.test_anomaly_post()
>             self.test_feature_get()
>             self.test_feature_put()
>
> Thanks for asking.
>
>
> On Tue, Aug 27, 2019 at 1:34 PM Chris Jerdonek <chris.jerdonek at gmail.com>
> wrote:
>
>> Why does each of your classes have a runTest method in the first place?
>> What is its purpose ?- what does it do?
>>
>> ?Chris
>>
>> On Tue, Aug 27, 2019 at 12:51 PM Dan Stromberg <drsalists at gmail.com>
>> wrote:
>>
>>>
>>> Hi folks.
>>>
>>> I'm working on a large Python 3/Python 2 codebase that has several
>>> automated test cases and test classes.  I know Python 2 is going away soon.
>>>
>>> We have lots of test* methods in classes that inherit
>>> from unittest.TestCase. Each of those classes also provides a runTest
>>> method.
>>>
>>> It's bugging me that the runTest method is getting used, and the test*
>>> methods are getting ignored, unless we call the test* methods from the
>>> runTest method.
>>>
>>> In fact, if I rename the runTest method from a test class to something
>>> unused, I get a traceback:
>>> runTest (tests.test_disposable_locks.TestDisposableLocks)
>>> No test ... Traceback (most recent call last):
>>>   File "test_runner.py", line 98, in <module>
>>>     sys.exit(main())
>>>   File "test_runner.py", line 90, in main
>>>     result = runner.run(test_suite)
>>>   File "/usr/lib/python3.6/unittest/runner.py", line 176, in run
>>>     test(result)
>>>   File "/usr/lib/python3.6/unittest/suite.py", line 84, in __call__
>>>     return self.run(*args, **kwds)
>>>   File "/usr/lib/python3.6/unittest/suite.py", line 122, in run
>>>     test(result)
>>>   File "/usr/lib/python3.6/unittest/case.py", line 653, in __call__
>>>     return self.run(*args, **kwds)
>>>   File "/usr/lib/python3.6/unittest/case.py", line 580, in run
>>>     testMethod = getattr(self, self._testMethodName)
>>> AttributeError: 'TestDisposableLocks' object has no attribute 'runTest'
>>>
>>> I've been pulling my hair out trying to figure out why unittest is
>>> preferring runTest over test*.  Isn't unittest supposed to fall back on
>>> runTest only if there are no test* methods?
>>>
>>> Or do I need something like:
>>> suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
>>> (from https://docs.python.org/2/library/unittest.html)
>>> ?  I tried getting this going with pytest a while back, but eventually
>>> set the project aside a while - maybe that was where I got the idea that
>>> runTest should be ignored if test* methods are present....
>>>
>>> I still hope to use pytest with this code at some point, but for now I'd
>>> be happy if the unittest test discovery just worked the way I expect.  My
>>> main goals for switching to pytest is to parallelise the tests, and to stop
>>> having to list "important" tests in a runTest method lest they be ignored.
>>>
>>> Also, I'm finding that sometimes raising an exception terminates the
>>> testing for the entire suite, rather than leading to a failed/errored test
>>> and continuing, presumably because "runTest" is "the test", so if it errors
>>> out anywhere, that's it for "that test".  If pytest can help with that,
>>> that'd be awesome.
>>>
>>> What can I do to get runTest out of the mix when test* methods are
>>> present?
>>>
>>> Thanks!
>>>
>>> _______________________________________________
>>> testing-in-python mailing list
>>> testing-in-python at lists.idyll.org
>>> http://lists.idyll.org/listinfo/testing-in-python
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190827/88bdd649/attachment-0001.htm>

From drsalists at gmail.com  Tue Aug 27 16:43:10 2019
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 27 Aug 2019 16:43:10 -0700
Subject: [TIP] runTest?
In-Reply-To: <CAOTb1wcQ3RXLco9KuAUZA+aE5JYHsUGY6HEa-OPBhgEMuh65rQ@mail.gmail.com>
References: <CAGGBd_raSLi84NEO0MNiMJFxG+sHk6YCcb195xTwtrgBBvb4iA@mail.gmail.com>
	<CAOTb1wf-r6k03OaqCCEGKZ55TeU2baBkT9+g8p5Dj2bnymTYHA@mail.gmail.com>
	<CAGGBd_pxsXVKL-Bd2d_c2niWgC6Yp9CbRa7DurFtfO5UgyimqQ@mail.gmail.com>
	<CAOTb1wcQ3RXLco9KuAUZA+aE5JYHsUGY6HEa-OPBhgEMuh65rQ@mail.gmail.com>
Message-ID: <CAGGBd_pEB8o-O_d-uu_=82UkGY1sqJDaT_o7EmfnqxuUdkMtgA@mail.gmail.com>

I'm inclined to agree with you - something unusual is happening in the code
on my end.

Sadly, I'm running out of ideas for where to look for the customization,
and the person who set it up is most likely no longer with the company.

Thanks though.

How about this: If someone were to try to set up such a thing, where tests
for a given test class are all ordered based on their appearance in a
single runTest method, how would you choose to go about setting that up?

We have a test_runner.py that controls our overall test execution, but it
doesn't have anything obviously runTest-related.

The closest thing that would be relevant in test_runner.py looks like:
    test_suite.addTest(TestDisposableLocks())

    # test_suite.addTest(TestTicketETL())

    runner = unittest.TextTestRunner(verbosity=3)
    result = runner.run(test_suite)
    if result.errors or result.failures:
        return 1
    else:
        return 0

I'm kind of wondering if TextTestRunner or runner.run is doing something
odd.  I've tried stepping into these with pudb, but it quickly turned into
a twisty maze.

Also, I wonder if one of the many pypi packages we're installing is doing
it.  Here are the ones that match the string 'test':
$ python -m pip freeze | grep -i test
below cmd output started 2019 Tue Aug 27 04:41:22 PM PDT
pytest==4.6.3
pytest-cov==2.5.0
pytest-forked==1.0.2
pytest-xdist==1.29.0
unittest2==0.5.1

Thanks.

On Tue, Aug 27, 2019 at 4:31 PM Chris Jerdonek <chris.jerdonek at gmail.com>
wrote:

> In my experience, I've never needed to add a runTest() method to TestCase
> classes, and if a TestCase class lacks any test methods, it will simply run
> no tests -- not try to run "runTest." If you look at unittest's source
> code, you'll see that "runTest" is the default test name if no method name
> is passed to TestCase's constructor. I'm guessing the behavior you're
> experiencing is due to a custom test runner or discovery or some other
> customization under your control. I've never seen this behavior from
> out-of-the-box uses of unittest.
>
> --Chris
>
>
> On Tue, Aug 27, 2019 at 1:40 PM Dan Stromberg <drsalists at gmail.com> wrote:
>
>>
>> It calls each "important" test in turn.  It's like a poor-person's
>> discovery. If a test is left out of runTest by accident, it gets ignored.
>>
>> Like:
>>     def runTest(self):
>>         assert self.test_red()
>>         assert self.test_green()
>>         assert self.test_blue()
>>
>> Or:
>>         def runTest(self):
>>             self.test_anomaly_post()
>>             self.test_feature_get()
>>             self.test_feature_put()
>>
>> Thanks for asking.
>>
>>
>> On Tue, Aug 27, 2019 at 1:34 PM Chris Jerdonek <chris.jerdonek at gmail.com>
>> wrote:
>>
>>> Why does each of your classes have a runTest method in the first place?
>>> What is its purpose ?- what does it do?
>>>
>>> ?Chris
>>>
>>> On Tue, Aug 27, 2019 at 12:51 PM Dan Stromberg <drsalists at gmail.com>
>>> wrote:
>>>
>>>>
>>>> Hi folks.
>>>>
>>>> I'm working on a large Python 3/Python 2 codebase that has several
>>>> automated test cases and test classes.  I know Python 2 is going away soon.
>>>>
>>>> We have lots of test* methods in classes that inherit
>>>> from unittest.TestCase. Each of those classes also provides a runTest
>>>> method.
>>>>
>>>> It's bugging me that the runTest method is getting used, and the test*
>>>> methods are getting ignored, unless we call the test* methods from the
>>>> runTest method.
>>>>
>>>> In fact, if I rename the runTest method from a test class to something
>>>> unused, I get a traceback:
>>>> runTest (tests.test_disposable_locks.TestDisposableLocks)
>>>> No test ... Traceback (most recent call last):
>>>>   File "test_runner.py", line 98, in <module>
>>>>     sys.exit(main())
>>>>   File "test_runner.py", line 90, in main
>>>>     result = runner.run(test_suite)
>>>>   File "/usr/lib/python3.6/unittest/runner.py", line 176, in run
>>>>     test(result)
>>>>   File "/usr/lib/python3.6/unittest/suite.py", line 84, in __call__
>>>>     return self.run(*args, **kwds)
>>>>   File "/usr/lib/python3.6/unittest/suite.py", line 122, in run
>>>>     test(result)
>>>>   File "/usr/lib/python3.6/unittest/case.py", line 653, in __call__
>>>>     return self.run(*args, **kwds)
>>>>   File "/usr/lib/python3.6/unittest/case.py", line 580, in run
>>>>     testMethod = getattr(self, self._testMethodName)
>>>> AttributeError: 'TestDisposableLocks' object has no attribute 'runTest'
>>>>
>>>> I've been pulling my hair out trying to figure out why unittest is
>>>> preferring runTest over test*.  Isn't unittest supposed to fall back on
>>>> runTest only if there are no test* methods?
>>>>
>>>> Or do I need something like:
>>>> suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
>>>> (from https://docs.python.org/2/library/unittest.html)
>>>> ?  I tried getting this going with pytest a while back, but eventually
>>>> set the project aside a while - maybe that was where I got the idea that
>>>> runTest should be ignored if test* methods are present....
>>>>
>>>> I still hope to use pytest with this code at some point, but for now
>>>> I'd be happy if the unittest test discovery just worked the way I expect.
>>>> My main goals for switching to pytest is to parallelise the tests, and to
>>>> stop having to list "important" tests in a runTest method lest they be
>>>> ignored.
>>>>
>>>> Also, I'm finding that sometimes raising an exception terminates the
>>>> testing for the entire suite, rather than leading to a failed/errored test
>>>> and continuing, presumably because "runTest" is "the test", so if it errors
>>>> out anywhere, that's it for "that test".  If pytest can help with that,
>>>> that'd be awesome.
>>>>
>>>> What can I do to get runTest out of the mix when test* methods are
>>>> present?
>>>>
>>>> Thanks!
>>>>
>>>> _______________________________________________
>>>> testing-in-python mailing list
>>>> testing-in-python at lists.idyll.org
>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190827/42c921a2/attachment.htm>

From chris.jerdonek at gmail.com  Tue Aug 27 17:11:53 2019
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Tue, 27 Aug 2019 17:11:53 -0700
Subject: [TIP] runTest?
In-Reply-To: <CAGGBd_pEB8o-O_d-uu_=82UkGY1sqJDaT_o7EmfnqxuUdkMtgA@mail.gmail.com>
References: <CAGGBd_raSLi84NEO0MNiMJFxG+sHk6YCcb195xTwtrgBBvb4iA@mail.gmail.com>
	<CAOTb1wf-r6k03OaqCCEGKZ55TeU2baBkT9+g8p5Dj2bnymTYHA@mail.gmail.com>
	<CAGGBd_pxsXVKL-Bd2d_c2niWgC6Yp9CbRa7DurFtfO5UgyimqQ@mail.gmail.com>
	<CAOTb1wcQ3RXLco9KuAUZA+aE5JYHsUGY6HEa-OPBhgEMuh65rQ@mail.gmail.com>
	<CAGGBd_pEB8o-O_d-uu_=82UkGY1sqJDaT_o7EmfnqxuUdkMtgA@mail.gmail.com>
Message-ID: <CAOTb1weXGZRSADbqDxNB-j2uRQxuthZtxT-XGPmC01JMcy1jKA@mail.gmail.com>

On Tue, Aug 27, 2019 at 4:43 PM Dan Stromberg <drsalists at gmail.com> wrote:

> The closest thing that would be relevant in test_runner.py looks like:
>     test_suite.addTest(TestDisposableLocks())
>

This is probably where it's happening. Here you're manually instantiating a
TestCase object, and since no "methodName" argument is being passed to the
constructor, it's defaulting to "runTest." (You can look at unittest's
source code to see this.)

--Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190827/df411a79/attachment.htm>

From cappy2112 at gmail.com  Tue Aug 27 21:45:27 2019
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Tue, 27 Aug 2019 21:45:27 -0700
Subject: [TIP] PyTest test directory structure confusion
Message-ID: <CABRXM4nb+TotU6g59Scj=a53+K6-jkwEaDATLtF37NA37wpHbA@mail.gmail.com>

Hi,

I?m new to using PyTest. I?ve got a very simple python file, and a very
simple test file for it.

I?ve been having a problem getting a unittest to be able to access the file
it?s trying to test. I?m sure that I don?t have the correct directory
configuration, but at the same time, I?m not sure what the correct
directory configuration should be though.

Page 25 of ?Python Testing with PyTest refers to __init__.py files in the
same directory as the test files themselves.

Under Project Structure, This website
https://automationpanda.com/2017/03/14/python-testing-101-pytest/

states that __init__.py files should NOT be in the same directory as the
test files.

__init__.py files have always been the biggest headache for me, as I?ve
never been able to find a clear reference as to what should  or shouldn?t
be in those files.
Why is it that some are empty, while others have tons of imports?


I would appreciate if someone could provide a reference as to how a PyTest
project should be structured, where __init__ files should and shouldn?t be,
along with their contents (if any).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190827/85d796ea/attachment-0001.htm>

From marius at gedmin.as  Wed Aug 28 03:24:00 2019
From: marius at gedmin.as (Marius Gedminas)
Date: Wed, 28 Aug 2019 13:24:00 +0300
Subject: [TIP] runTest?
In-Reply-To: <CAOTb1weXGZRSADbqDxNB-j2uRQxuthZtxT-XGPmC01JMcy1jKA@mail.gmail.com>
References: <CAGGBd_raSLi84NEO0MNiMJFxG+sHk6YCcb195xTwtrgBBvb4iA@mail.gmail.com>
	<CAOTb1wf-r6k03OaqCCEGKZ55TeU2baBkT9+g8p5Dj2bnymTYHA@mail.gmail.com>
	<CAGGBd_pxsXVKL-Bd2d_c2niWgC6Yp9CbRa7DurFtfO5UgyimqQ@mail.gmail.com>
	<CAOTb1wcQ3RXLco9KuAUZA+aE5JYHsUGY6HEa-OPBhgEMuh65rQ@mail.gmail.com>
	<CAGGBd_pEB8o-O_d-uu_=82UkGY1sqJDaT_o7EmfnqxuUdkMtgA@mail.gmail.com>
	<CAOTb1weXGZRSADbqDxNB-j2uRQxuthZtxT-XGPmC01JMcy1jKA@mail.gmail.com>
Message-ID: <20190828102400.loz5ljo422czr7hh@blynas>

On Tue, Aug 27, 2019 at 05:11:53PM -0700, Chris Jerdonek wrote:
> On Tue, Aug 27, 2019 at 4:43 PM Dan Stromberg <[1]drsalists at gmail.com> wrote:
> 
>     The closest thing that would be relevant in test_runner.py looks like:
>     ? ? test_suite.addTest(TestDisposableLocks())
> 
> 
> This is probably where it's happening. Here you're manually instantiating a
> TestCase object, and since no "methodName" argument is being passed to the
> constructor, it's defaulting to "runTest." (You can look at unittest's source
> code to see this.)

Right, and to make it actually pay attention to 'test*' methods in the
class, what you're supposed to do is

        test_suite.addTest(unittest.makeSuite(TestDisposableLocks))

if you want to continue enumerating all the test classes by hand.  Or
you could use a test loader to discover all the test suite classes and
create test suites for them for you.

(Under the hood what makeSuite does is instantiate TestDisposableLocks
multiple times, passing each test name to the constructor.)

Marius Gedminas
-- 
The most straightforward thing to assume was that the BIOS was stupid (which
is, to be fair, my default assumption) and shouldn't have enabled ASPM.
        -- Matthew Garrett
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190828/4522c1c2/attachment.pgp>

From marius at gedmin.as  Wed Aug 28 04:48:21 2019
From: marius at gedmin.as (Marius Gedminas)
Date: Wed, 28 Aug 2019 14:48:21 +0300
Subject: [TIP] PyTest test directory structure confusion
In-Reply-To: <CABRXM4nb+TotU6g59Scj=a53+K6-jkwEaDATLtF37NA37wpHbA@mail.gmail.com>
References: <CABRXM4nb+TotU6g59Scj=a53+K6-jkwEaDATLtF37NA37wpHbA@mail.gmail.com>
Message-ID: <20190828114821.zo5dt4bmjxgcs6ss@blynas>

Hi!

On Tue, Aug 27, 2019 at 09:45:27PM -0700, Tony Cappellini wrote:
> I?m new to using PyTest. I?ve got a very simple python file, and a very simple
> test file for it.
...

> I?ve been having a problem getting a unittest to be able to access the file
> it?s trying to test. I?m sure that I don?t have the correct directory
> configuration, but at the same time, I?m not sure what the correct directory
> configuration should be though.

You have the simplest possible case: one code file, one test file.  You
don't need a directory structure at this point, just the two files:

    mycode.py
    tests.py

inside your tests.py you can do

    import mycode

    def test_my_stuff():
        assert mycode.stuff(42) == 25   # etc.

and you should be able to run the tests with

    pytest tests.py

If you don't want to keep repeating 'tests.py' on the command line,
create a pytest.ini with

    [pytest]
    testpaths = tests.py

and then you can run just

    pytest

That's it!

I use this structure for a few of my small Python projects, e.g.
https://github.com/mgedmin/ghcloneall


> Page 25 of ?Python Testing with PyTest refers to __init__.py files in the same
> directory as the test files themselves.
> 
> Under Project Structure, This website?
> [1]https://automationpanda.com/2017/03/14/python-testing-101-pytest/
> 
> states that __init__.py files should NOT be in the same directory as the test
> files.

I've no idea why that page makes this claim.  I have my tests in a
package in https://github.com/mgedmin/pyspacewar, and pytest has no
trouble with that structure.

> __init__.py files have always been the biggest headache for me, as I?ve never
> been able to find a clear reference as to what should or shouldn?t be in those
> files.
> Why is it that some are empty, while others have tons of imports?

I assume you know what they're for?  If not, see the Python Tutorial
section on packages: https://docs.python.org/3/tutorial/modules.html#packages

As for the contents, the short answer is: if you're not sure, keep
__init__.py files empty.  But if you feel like you want to put some code
in there, there likely won't be any harm.

> I would appreciate if someone could provide a reference as to how a PyTest
> project should be structured, where __init__ files should and shouldn?t be,
> along with their contents (if any).

There's more than one common source layout, and which is best is a
matter of opinion.


The one I'm used to has a 'src/' directory at the top, and puts all the
tests as subpackages inside the actual code tree.  Like, imagine if the
example I gave before grew and grew and became too big for a single
source file:

    mycode.py
    tests.py
    pytest.ini
    setup.py
    tox.ini

I've added a setup.py because you probably want to be able to pip
install the thing, and pull it any dependencies it has.  I've added a
tox.ini because tox is a wonderful tool for automating the tests -- it
creates virtualenvs for you so you get all the code and test
dependencies without polluting your global Python installation and
interfering with other projects.

(For completeness, here's what a setup.py looks:

    from setuptools import setup
    setup(
        name='mycode',
        version=...,
        author=...,
        license=...,
        url=...,
        description=...,
        long_description=...,
        py_modules=['mycode'],
    )

and here's tox.ini:

    [tox]
    envlist = py37

    [testenv]
    deps = pytest
    commands = pytest {posargs}

)

So, at this point splitting the code into several modules will result in

    src/
        mycode/
            __init__.py                    # maybe empty, maybe not
            things.py
            more_things.py
            tests/
                __init__.py                # empty file
                test_mycode.py
                test_things.py
                test_more_things.py
    pytest.ini
    setup.py
    tox.ini

Here each code module has a corresponding test module, for my own
convenience: when I want to find the unit tests for a function
mycode.things.do_x(), I'll know to look in mycode.tests.test_things
for test functions called test_do_x (with optional suffixes if there's
more than one test).

To make this work, pytest.ini changes from

    [pytest]
    testpaths = tests.py

to

    [pytest]
    testpaths = src

and you can't run 'pytest' without arguments any more (because ./src is
not in PYTHONPATH -- this is the downside of using a top-level 'src'
subdirectory), but you can run 'tox' without arguments.  tox.ini is
unchanged, setup.py changes from using

    from setuptools import setup
    setup(
        ...
        py_modules='mycode',
    )

to

    from setuptools import setup, find_packages
    setup(
        ...
        packages=find_packages('src'),
        package_dir={'': 'src'},
    )

Now, if you want to continue to write code like

    from mycode import stuff

then mycode/__init__.py will have to define the 'stuff' function, or
import and re-export it like this:

    # mycode/__init__.py
    from .things import stuff   # reexport

But if you're fine requiring that all users of mycode change their code
to do

    from mycode.things import stuff

then mycode/__init__.py can be an empty file.


Hope that helped!

Marius Gedminas
-- 
/*
 * The lockdep graph lock isn't locked while we expect it to
 * be, we're confused now, bye!
 */
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190828/81cb52f6/attachment.pgp>

From variedthoughts at gmail.com  Wed Aug 28 08:11:37 2019
From: variedthoughts at gmail.com (Brian Okken)
Date: Wed, 28 Aug 2019 08:11:37 -0700
Subject: [TIP] PyTest test directory structure confusion
In-Reply-To: <20190828114821.zo5dt4bmjxgcs6ss@blynas>
References: <CABRXM4nb+TotU6g59Scj=a53+K6-jkwEaDATLtF37NA37wpHbA@mail.gmail.com>
	<20190828114821.zo5dt4bmjxgcs6ss@blynas>
Message-ID: <CAHsuHJ2xSA=RP3S0dDyC7qz6R1d8vSYUWb0jxHEHW7YCrCU6_g@mail.gmail.com>

If you have all of your tests under a "tests" directory separate from your
source code, you generally don't need __init__.py files.
The pytest book goes into more detail about it in chapter 6 under the
section "Avoiding Filename Collisions".
If you have two test files, or think you might someday, that are the same
name in different subdirectories, then the __init__.py technique is useful
to remove the collision.
These files are empty.

So, under simple cases, you don't need __init__.py files.
If you have them, they won't hurt, and might help.

Hope this helps.
- brian

- Brian

On Wed, Aug 28, 2019 at 4:52 AM Marius Gedminas <marius at gedmin.as> wrote:

> Hi!
>
> On Tue, Aug 27, 2019 at 09:45:27PM -0700, Tony Cappellini wrote:
> > I?m new to using PyTest. I?ve got a very simple python file, and a very
> simple
> > test file for it.
> ...
>
> > I?ve been having a problem getting a unittest to be able to access the
> file
> > it?s trying to test. I?m sure that I don?t have the correct directory
> > configuration, but at the same time, I?m not sure what the correct
> directory
> > configuration should be though.
>
> You have the simplest possible case: one code file, one test file.  You
> don't need a directory structure at this point, just the two files:
>
>     mycode.py
>     tests.py
>
> inside your tests.py you can do
>
>     import mycode
>
>     def test_my_stuff():
>         assert mycode.stuff(42) == 25   # etc.
>
> and you should be able to run the tests with
>
>     pytest tests.py
>
> If you don't want to keep repeating 'tests.py' on the command line,
> create a pytest.ini with
>
>     [pytest]
>     testpaths = tests.py
>
> and then you can run just
>
>     pytest
>
> That's it!
>
> I use this structure for a few of my small Python projects, e.g.
> https://github.com/mgedmin/ghcloneall
>
>
> > Page 25 of ?Python Testing with PyTest refers to __init__.py files in
> the same
> > directory as the test files themselves.
> >
> > Under Project Structure, This website
> > [1]https://automationpanda.com/2017/03/14/python-testing-101-pytest/
> >
> > states that __init__.py files should NOT be in the same directory as the
> test
> > files.
>
> I've no idea why that page makes this claim.  I have my tests in a
> package in https://github.com/mgedmin/pyspacewar, and pytest has no
> trouble with that structure.
>
> > __init__.py files have always been the biggest headache for me, as I?ve
> never
> > been able to find a clear reference as to what should or shouldn?t be in
> those
> > files.
> > Why is it that some are empty, while others have tons of imports?
>
> I assume you know what they're for?  If not, see the Python Tutorial
> section on packages:
> https://docs.python.org/3/tutorial/modules.html#packages
>
> As for the contents, the short answer is: if you're not sure, keep
> __init__.py files empty.  But if you feel like you want to put some code
> in there, there likely won't be any harm.
>
> > I would appreciate if someone could provide a reference as to how a
> PyTest
> > project should be structured, where __init__ files should and shouldn?t
> be,
> > along with their contents (if any).
>
> There's more than one common source layout, and which is best is a
> matter of opinion.
>
>
> The one I'm used to has a 'src/' directory at the top, and puts all the
> tests as subpackages inside the actual code tree.  Like, imagine if the
> example I gave before grew and grew and became too big for a single
> source file:
>
>     mycode.py
>     tests.py
>     pytest.ini
>     setup.py
>     tox.ini
>
> I've added a setup.py because you probably want to be able to pip
> install the thing, and pull it any dependencies it has.  I've added a
> tox.ini because tox is a wonderful tool for automating the tests -- it
> creates virtualenvs for you so you get all the code and test
> dependencies without polluting your global Python installation and
> interfering with other projects.
>
> (For completeness, here's what a setup.py looks:
>
>     from setuptools import setup
>     setup(
>         name='mycode',
>         version=...,
>         author=...,
>         license=...,
>         url=...,
>         description=...,
>         long_description=...,
>         py_modules=['mycode'],
>     )
>
> and here's tox.ini:
>
>     [tox]
>     envlist = py37
>
>     [testenv]
>     deps = pytest
>     commands = pytest {posargs}
>
> )
>
> So, at this point splitting the code into several modules will result in
>
>     src/
>         mycode/
>             __init__.py                    # maybe empty, maybe not
>             things.py
>             more_things.py
>             tests/
>                 __init__.py                # empty file
>                 test_mycode.py
>                 test_things.py
>                 test_more_things.py
>     pytest.ini
>     setup.py
>     tox.ini
>
> Here each code module has a corresponding test module, for my own
> convenience: when I want to find the unit tests for a function
> mycode.things.do_x(), I'll know to look in mycode.tests.test_things
> for test functions called test_do_x (with optional suffixes if there's
> more than one test).
>
> To make this work, pytest.ini changes from
>
>     [pytest]
>     testpaths = tests.py
>
> to
>
>     [pytest]
>     testpaths = src
>
> and you can't run 'pytest' without arguments any more (because ./src is
> not in PYTHONPATH -- this is the downside of using a top-level 'src'
> subdirectory), but you can run 'tox' without arguments.  tox.ini is
> unchanged, setup.py changes from using
>
>     from setuptools import setup
>     setup(
>         ...
>         py_modules='mycode',
>     )
>
> to
>
>     from setuptools import setup, find_packages
>     setup(
>         ...
>         packages=find_packages('src'),
>         package_dir={'': 'src'},
>     )
>
> Now, if you want to continue to write code like
>
>     from mycode import stuff
>
> then mycode/__init__.py will have to define the 'stuff' function, or
> import and re-export it like this:
>
>     # mycode/__init__.py
>     from .things import stuff   # reexport
>
> But if you're fine requiring that all users of mycode change their code
> to do
>
>     from mycode.things import stuff
>
> then mycode/__init__.py can be an empty file.
>
>
> Hope that helped!
>
> Marius Gedminas
> --
> /*
>  * The lockdep graph lock isn't locked while we expect it to
>  * be, we're confused now, bye!
>  */
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190828/77d798e7/attachment-0001.htm>

From cappy2112 at gmail.com  Thu Aug 29 14:40:24 2019
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Thu, 29 Aug 2019 14:40:24 -0700
Subject: [TIP] Running PyTest with coverage from PyCharm
Message-ID: <CABRXM4=dgRea1w5WpyBsvH9Th52Yu3Y8mJ+3=Qbz1pj3hLQ4_w@mail.gmail.com>

I've managed to get Pytest to run my test file.

I then thought I would run Pytest with Coverage from within Pycharm
(because I couldn't find the information on how to run pytest & coverage at
the same time, from the command line. The docs on this are somewhat
vague.).

Within Pycharm, you can tell Pytest exactly where to run, if you want to
focus on a specific test or test directory. However, there doesn't appear
to be an option to tell Pycharm to run coverage to run from the same
directory as the test file.

The file being tested is fairly short (about 20 lines). The test file I've
written is also fairly short, about 20-25 lines across 3 test functions.
I'm expecting that my tests are covering a fairly high percentage of the
file being tested. Running coverage from Pycharm shows this: 6% files, 84%
lines covered.

The 6% comes to me as a surprise since I'm only testing 1 file with 1 test
file.
Looking into the coverage docs, it looks like that I can tell coverage
where to un, include, and exclude using .coveragerc. So I've added a
specific path to the directory where my test file is. Nothing changes in
the coverage results.

I then tried running coverage from the command line, and am seeing an
entirely different set of coverage statistics.

----------- coverage: platform win32, python 3.6.6-final-0 -----------
Name                                Stmts   Miss  Cover
-------------------------------------------------------
unit_tests\logfile.py                  37      7    81%
unit_tests\never_stops_running.py       7      7     0%
unit_tests\test_popen.py               29      0   100%
-------------------------------------------------------
TOTAL                                  73     14    81%


*My questions:*
1. What needs to change so that I get the same results when running
Coverage from the command line and within Pycharm?
2. When running Coverage from Pycharm is there a way to tell coverage to
show me which lines are not being tested?
3. Does anyone else feel that PyCharm should have a dialog that allows the
user to point it to which code to measure coverage on? The Run/Debug dialog
gives the user control over where to run Pytest, so why not coverage too?

thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190829/67bc69a3/attachment.htm>

From cappy2112 at gmail.com  Thu Aug 29 14:56:22 2019
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Thu, 29 Aug 2019 14:56:22 -0700
Subject: [TIP] PyTest test directory structure confusion
In-Reply-To: <mailman.2.1567005109.9269.testing-in-python@lists.idyll.org>
References: <mailman.2.1567005109.9269.testing-in-python@lists.idyll.org>
Message-ID: <CABRXM4kT3kigBHGsZvCiTRDz=2-Q8=meuPFRgLs2P3hqks-oCA@mail.gmail.com>

Message: 2
Date: Wed, 28 Aug 2019 14:48:21 +0300
From: Marius Gedminas <marius at gedmin.as>
Subject: Re: [TIP] PyTest test directory structure confusion
To: testing-in-python at lists.idyll.org
Message-ID: <20190828114821.zo5dt4bmjxgcs6ss at blynas>
Content-Type: text/plain; charset="utf-8"

Thanks Marius

>>You have the simplest possible case: one code file, one test file.  You
>>don't need a directory structure at this point, just the two files:

>>    mycode.py
>>    tests.py

Even though I'm just now getting my head around testing with Pytest,
I've decided to put the tests in a separate directory called unit_tests,
which is at the same level as the directory containing the file being
tested.

I don't have any __init__.py in the test directory.

I don't know if my setup is optimal yet, but I'm able to run the tests I've
written and got reasonable results. Getting tests written and able to run
is a significant first step, even though there are still many questions in
my head.
The main thing is that I'm testing- finally.

Thanks- all
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190829/e0a225e4/attachment.htm>

From techtonik at gmail.com  Sat Aug 31 00:39:43 2019
From: techtonik at gmail.com (anatoly techtonik)
Date: Sat, 31 Aug 2019 10:39:43 +0300
Subject: [TIP] No boilerplate testing framework
Message-ID: <CAPkN8xLbLw9siTw_xKfkBo2nmZnk=BVpRy+ZZrBFUBgG=SZwqg@mail.gmail.com>

I've got a simple Python module named **pypack.py**, and I want to
test it with as little boilerplate as possible. I want to write my
tests in **pypack_test.py**  (similar to Go naming convention). And I
want these tests to be written like this.

    assert_equals(get_description("Short"), "Short")
    assert_equals(get_description("Short\n\nLong"), "Short")

Notes:

1. `get_description` is automatically imported from `pypack`
2. `assert_equals` is automatically inserted by test harness

Are there any frameworks that could be executed from terminal as a
single command that allow just that? If not, how hard it is to
implement?

On separate a wishlist is (1) detection of conflicting symbols between
test harness and tested module, (2) automated timings by detecting
test execution boundaries.
-- 
anatoly t.