[TIP] runTest?
Dan Stromberg
drsalists at gmail.com
Tue Aug 27 16:43:10 PDT 2019
I'm inclined to agree with you - something unusual is happening in the code
on my end.
Sadly, I'm running out of ideas for where to look for the customization,
and the person who set it up is most likely no longer with the company.
Thanks though.
How about this: If someone were to try to set up such a thing, where tests
for a given test class are all ordered based on their appearance in a
single runTest method, how would you choose to go about setting that up?
We have a test_runner.py that controls our overall test execution, but it
doesn't have anything obviously runTest-related.
The closest thing that would be relevant in test_runner.py looks like:
test_suite.addTest(TestDisposableLocks())
# test_suite.addTest(TestTicketETL())
runner = unittest.TextTestRunner(verbosity=3)
result = runner.run(test_suite)
if result.errors or result.failures:
return 1
else:
return 0
I'm kind of wondering if TextTestRunner or runner.run is doing something
odd. I've tried stepping into these with pudb, but it quickly turned into
a twisty maze.
Also, I wonder if one of the many pypi packages we're installing is doing
it. Here are the ones that match the string 'test':
$ python -m pip freeze | grep -i test
below cmd output started 2019 Tue Aug 27 04:41:22 PM PDT
pytest==4.6.3
pytest-cov==2.5.0
pytest-forked==1.0.2
pytest-xdist==1.29.0
unittest2==0.5.1
Thanks.
On Tue, Aug 27, 2019 at 4:31 PM Chris Jerdonek <chris.jerdonek at gmail.com>
wrote:
> In my experience, I've never needed to add a runTest() method to TestCase
> classes, and if a TestCase class lacks any test methods, it will simply run
> no tests -- not try to run "runTest." If you look at unittest's source
> code, you'll see that "runTest" is the default test name if no method name
> is passed to TestCase's constructor. I'm guessing the behavior you're
> experiencing is due to a custom test runner or discovery or some other
> customization under your control. I've never seen this behavior from
> out-of-the-box uses of unittest.
>
> --Chris
>
>
> On Tue, Aug 27, 2019 at 1:40 PM Dan Stromberg <drsalists at gmail.com> wrote:
>
>>
>> It calls each "important" test in turn. It's like a poor-person's
>> discovery. If a test is left out of runTest by accident, it gets ignored.
>>
>> Like:
>> def runTest(self):
>> assert self.test_red()
>> assert self.test_green()
>> assert self.test_blue()
>>
>> Or:
>> def runTest(self):
>> self.test_anomaly_post()
>> self.test_feature_get()
>> self.test_feature_put()
>>
>> Thanks for asking.
>>
>>
>> On Tue, Aug 27, 2019 at 1:34 PM Chris Jerdonek <chris.jerdonek at gmail.com>
>> wrote:
>>
>>> Why does each of your classes have a runTest method in the first place?
>>> What is its purpose —- what does it do?
>>>
>>> —Chris
>>>
>>> On Tue, Aug 27, 2019 at 12:51 PM Dan Stromberg <drsalists at gmail.com>
>>> wrote:
>>>
>>>>
>>>> Hi folks.
>>>>
>>>> I'm working on a large Python 3/Python 2 codebase that has several
>>>> automated test cases and test classes. I know Python 2 is going away soon.
>>>>
>>>> We have lots of test* methods in classes that inherit
>>>> from unittest.TestCase. Each of those classes also provides a runTest
>>>> method.
>>>>
>>>> It's bugging me that the runTest method is getting used, and the test*
>>>> methods are getting ignored, unless we call the test* methods from the
>>>> runTest method.
>>>>
>>>> In fact, if I rename the runTest method from a test class to something
>>>> unused, I get a traceback:
>>>> runTest (tests.test_disposable_locks.TestDisposableLocks)
>>>> No test ... Traceback (most recent call last):
>>>> File "test_runner.py", line 98, in <module>
>>>> sys.exit(main())
>>>> File "test_runner.py", line 90, in main
>>>> result = runner.run(test_suite)
>>>> File "/usr/lib/python3.6/unittest/runner.py", line 176, in run
>>>> test(result)
>>>> File "/usr/lib/python3.6/unittest/suite.py", line 84, in __call__
>>>> return self.run(*args, **kwds)
>>>> File "/usr/lib/python3.6/unittest/suite.py", line 122, in run
>>>> test(result)
>>>> File "/usr/lib/python3.6/unittest/case.py", line 653, in __call__
>>>> return self.run(*args, **kwds)
>>>> File "/usr/lib/python3.6/unittest/case.py", line 580, in run
>>>> testMethod = getattr(self, self._testMethodName)
>>>> AttributeError: 'TestDisposableLocks' object has no attribute 'runTest'
>>>>
>>>> I've been pulling my hair out trying to figure out why unittest is
>>>> preferring runTest over test*. Isn't unittest supposed to fall back on
>>>> runTest only if there are no test* methods?
>>>>
>>>> Or do I need something like:
>>>> suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
>>>> (from https://docs.python.org/2/library/unittest.html)
>>>> ? I tried getting this going with pytest a while back, but eventually
>>>> set the project aside a while - maybe that was where I got the idea that
>>>> runTest should be ignored if test* methods are present....
>>>>
>>>> I still hope to use pytest with this code at some point, but for now
>>>> I'd be happy if the unittest test discovery just worked the way I expect.
>>>> My main goals for switching to pytest is to parallelise the tests, and to
>>>> stop having to list "important" tests in a runTest method lest they be
>>>> ignored.
>>>>
>>>> Also, I'm finding that sometimes raising an exception terminates the
>>>> testing for the entire suite, rather than leading to a failed/errored test
>>>> and continuing, presumably because "runTest" is "the test", so if it errors
>>>> out anywhere, that's it for "that test". If pytest can help with that,
>>>> that'd be awesome.
>>>>
>>>> What can I do to get runTest out of the mix when test* methods are
>>>> present?
>>>>
>>>> Thanks!
>>>>
>>>> _______________________________________________
>>>> testing-in-python mailing list
>>>> testing-in-python at lists.idyll.org
>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190827/42c921a2/attachment.htm>
More information about the testing-in-python
mailing list