[TIP] "Re: Contents of testing-in-python Digest, Vol 69, Issue 4
Dan Wandschneider
daniel.wandschneider at schrodinger.com
Wed Oct 3 13:02:01 PDT 2012
Ned-
I appreciate that you highlighted the assertRaises method. I think
that it is very important and underused.
-Dan W.
On Wed, Oct 3, 2012 at 12:00 PM,
<testing-in-python-request at lists.idyll.org> wrote:
> Send testing-in-python mailing list submissions to
> testing-in-python at lists.idyll.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.idyll.org/listinfo/testing-in-python
> or, via email, send a message with subject or body 'help' to
> testing-in-python-request at lists.idyll.org
>
> You can reach the person managing the list at
> testing-in-python-owner at lists.idyll.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of testing-in-python digest..."
>
> Today's Topics:
>
> 1. Re: structure of a testing talk (Ned Batchelder)
> 2. Re: structure of a testing talk (Mark Sienkiewicz)
> 3. Re: how to access the test discovery protocol
> programmatically in pytest (Ronny Pfannschmidt)
> 4. Re: how to access the test discovery protocol
> programmatically in pytest (Pella,Chris)
> 5. Re: structure of a testing talk (Andrea Crotti)
>
>
> ---------- Forwarded message ----------
> From: Ned Batchelder <ned at nedbatchelder.com>
> To: andrea crotti <andrea.crotti.0 at gmail.com>
> Cc: testing-in-python at lists.idyll.org
> Date: Wed, 03 Oct 2012 07:03:59 -0400
> Subject: Re: [TIP] structure of a testing talk
>
> On 10/2/2012 8:08 AM, andrea crotti wrote:
>>
>> 2012/10/2 Ned Batchelder <ned at nedbatchelder.com>:
>>>
>>>
>>> Andrea, this is helpful to me. I've proposed a Getting Started Testing
>>> tutorial for PyCon, and it's great to see how other people tell the story.
>>> My tutorial is an expansion of a talk I did last year at Boston Python. You
>>> can see my slides from that talk at
>>> http://nedbatchelder.com/text/starttest.html . I'll accept tweaks to my
>>> story line too!
>>>
>>> --Ned.
>>
>>
>> It looks very nice thanks, I like the idea of evolving tests to show
>> how it's superior to doing it by hand..
>>
>> But are also the sources for the slides available? I think you might
>> really like https://github.com/nyergler/hieroglyph to produce html
>> slides, the output is great and you can use all the Sphinx plugins (I
>> use ditaa / graphviz and all the source code smart inclusions for
>> example).
>
> The HTML file is the source. I use Cog to add automatic content to it, so the file is both the source and the output. I'll take a look at hieroglyph, thanks.
>
> --Ned.
>
>> Here there is another example of two talks I've with this method:
>> https://github.com/AndreaCrotti/pyconuk2012_slides
>>
>> Anyway only things I found are
>> - slide 11 goes out
>> - colors are a bit "shocking"
>>
>> Another nice thing which I've seen doing once (but might not be always
>> possible) is to actually do some Test First Development calling people
>> from the audience, each one creating a test and make it pass.
>>
>> It's a bit slow but a great way to make sure you're not going too fast
>> and to see how someone not experience approaches the problem..
>>
>
>
>
>
>
> ---------- Forwarded message ----------
> From: Mark Sienkiewicz <sienkiew at stsci.edu>
> To: <testing-in-python at lists.idyll.org>
> Cc:
> Date: Wed, 3 Oct 2012 12:59:15 -0400
> Subject: Re: [TIP] structure of a testing talk
> On 10/03/12 05:14, andrea crotti wrote:
>>
>> Well not exactly, I'm speaking to experienced C developers, so probably not
>> worth to talk about exoteric things ;) One thing which I think is important
>> is to explain that the important skill is not much writing tests, but
>> writing code that it's easy to test..
>>
>
>
> The buzzword is "Design for Test", and it is a big deal in hardware. I suggest you give it a top-level bullet point in your presentation. Instead of mentioning it at the end, mention it at the beginning and use it in your examples. You want to be talking about something else and then then say "by the way, see how this thing I am testing was designed to be easy to test?".
>
> If you can, include at least one example where the software would be hard to test, except that it has a specific test featurein it. (Yes, it's hard to make a simple example.)
>
>
> b.t.w. In your outline, the one thing that stands out for me is "Side effects, and why they are bad". I think you might lose a lot of C programmers by saying it that way, because a lot of C code exists _specifically_ _for_ the side effects. write() is only useful because of the side-effect, for example, and we have exactly the same function in python. Saying "side effects are bad" may turn off your audience and make them less able/willing to listen to the rest of what you have to say.
>
> In your presentation, funcs.silly_function() appears to generate blocks of serial numbers (though you are only asking for blocks containing 1 serial number in your example). Can you explain to a C programmer what exactly is bad about a function that returns a unique serial number every time you call it? I wouldn't bother talking about a singleton serial number object, because a C programmer will immediately recognize that as the same thing.
>
> Instead, I think you want to talk about appropriateness of side effects. It can be easy to pack weird side-effects into all your functions; then you have a system that is hard to understand. Narrowly, carefully, and clearly defined side effects are good; random, arbitrary, or confusing side effects are bad.
>
> You can integrate this idea into the design-for-test model. When you have side effects, you have to have a way to determine if they happened. This means both an interface to peek into <whatever> to see if the side effect happened, and a clear definition of what was _supposed_ to happen. You can be motivated to write more purely functional code because it will be so much easier to understand and to test.
>
> Mark S.
>
>
>
>
>
> ---------- Forwarded message ----------
> From: Ronny Pfannschmidt <Ronny.Pfannschmidt at gmx.de>
> To: "Pella,Chris" <Chris.Pella at safenet-inc.com>
> Cc: "testing-in-python at lists.idyll.org" <testing-in-python at lists.idyll.org>
> Date: Wed, 03 Oct 2012 19:54:04 +0200
> Subject: Re: [TIP] how to access the test discovery protocol programmatically in pytest
> Hi Chris,
>
> you can just do what the terminal plugin does
>
> the pytest_collection_finish(session) hook can access session.items
>
> session.items is are the actual tests (i.e. the leaves of the collection tree)
>
> note that for finding cases where collection fails you will also need the pytest_collectreport(report) hook
>
>
> if you use pytest.main you can just pass in a object with methods that follow the call signatures i have (can be a module or a instance)
>
> -- Ronny
>
> On 10/02/2012 10:29 PM, Pella,Chris wrote:
>>
>> We are calling pytest.main to run tests from our own home-grown
>> framework. I understand this is supposed to be an api-less framework,
>> but is there a way to access the equivalent results to running py.test
>> –collectonly other than redirecting the output to a file and reading the
>> file?
>>
>> Thanks,
>>
>> Chris
>>
>> The information contained in this electronic mail transmission
>> may be privileged and confidential, and therefore, protected
>> from disclosure. If you have received this communication in
>> error, please notify us immediately by replying to this
>> message and deleting it from your computer without copying
>> or disclosing it.
>>
>>
>>
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>
>
>
>
>
>
> ---------- Forwarded message ----------
> From: "Pella,Chris" <Chris.Pella at safenet-inc.com>
> To: Ronny Pfannschmidt <Ronny.Pfannschmidt at gmx.de>
> Cc: "testing-in-python at lists.idyll.org" <testing-in-python at lists.idyll.org>
> Date: Wed, 3 Oct 2012 14:17:43 -0400
> Subject: Re: [TIP] how to access the test discovery protocol programmatically in pytest
> Ronny,
> Thanks a lot.
>
> Chris
>
> -----Original Message-----
> From: Ronny Pfannschmidt [mailto:Ronny.Pfannschmidt at gmx.de]
> Sent: Wednesday, October 03, 2012 1:54 PM
> To: Pella,Chris
> Cc: testing-in-python at lists.idyll.org
> Subject: Re: [TIP] how to access the test discovery protocol programmatically in pytest
>
> Hi Chris,
>
> you can just do what the terminal plugin does
>
> the pytest_collection_finish(session) hook can access session.items
>
> session.items is are the actual tests (i.e. the leaves of the collection
> tree)
>
> note that for finding cases where collection fails you will also need
> the pytest_collectreport(report) hook
>
>
> if you use pytest.main you can just pass in a object with methods that
> follow the call signatures i have (can be a module or a instance)
>
> -- Ronny
>
> On 10/02/2012 10:29 PM, Pella,Chris wrote:
>> We are calling pytest.main to run tests from our own home-grown
>> framework. I understand this is supposed to be an api-less framework,
>> but is there a way to access the equivalent results to running py.test
>> -collectonly other than redirecting the output to a file and reading the
>> file?
>>
>> Thanks,
>>
>> Chris
>>
>> The information contained in this electronic mail transmission
>> may be privileged and confidential, and therefore, protected
>> from disclosure. If you have received this communication in
>> error, please notify us immediately by replying to this
>> message and deleting it from your computer without copying
>> or disclosing it.
>>
>>
>>
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>
> The information contained in this electronic mail transmission
> may be privileged and confidential, and therefore, protected
> from disclosure. If you have received this communication in
> error, please notify us immediately by replying to this
> message and deleting it from your computer without copying
> or disclosing it.
>
>
>
>
>
>
>
> ---------- Forwarded message ----------
> From: Andrea Crotti <andrea.crotti.0 at gmail.com>
> To: Mark Sienkiewicz <sienkiew at stsci.edu>
> Cc: testing-in-python at lists.idyll.org
> Date: Wed, 03 Oct 2012 19:50:01 +0100
> Subject: Re: [TIP] structure of a testing talk
> 10/03/2012 05:59 PM, Mark Sienkiewicz wrote:
>>
>>
>>
>> The buzzword is "Design for Test", and it is a big deal in hardware. I suggest you give it a top-level bullet point in your presentation. Instead of mentioning it at the end, mention it at the beginning and use it in your examples. You want to be talking about something else and then then say "by the way, see how this thing I am testing was designed to be easy to test?".
>>
>> If you can, include at least one example where the software would be hard to test, except that it has a specific test featurein it. (Yes, it's hard to make a simple theb.t.w. In your outline, the one thing that stands out for me is "Side effects, and why they are bad". I think you might lose a lot of C programmers by saying it that way, because a lot of C code exists _specifically_ _for_ the side effects. write() is only useful because of the side-effect, for example, and we have exactly the same function in python. Saying "side effects are bad" may turn off your audience and make them less able/willing to listen to the rest of what you have to say.
>>
>> In your presentation, funcs.silly_function() appears to generate blocks of serial numbers (though you are only asking for blocks containing 1 serial number in your example). Can you explain to a C programmer what exactly is bad about a function that returns a unique serial number every time you call it? I wouldn't bother talking about a singleton serial number object, because a C programmer will immediately recognize that as the same thing.
>>
>> Instead, I think you want to talk about appropriateness of side effects. It can be easy to pack weird side-effects into all your functions; then you have a system that is hard to understand. Narrowly, carefully, and clearly defined side effects are good; random, arbitrary, or confusing side effects are bad.
>>
>> You can integrate this idea into the design-for-test model. When you have side effects, you have to have a way to determine if they happened. This means both an interface to peek into <whatever> to see if the side effect happened, and a clear definition of what was _supposed_ to happen. You can be motivated to write more purely functional code because it will be so much easier to understand and to test.
>>
>> Mark S.
>
>
>
> Thanks a lot for the feedback, I will write more about this tomorrow..
> But in general I think side effects are bad also in C, of course it's harder but in my projects
> I tried to be as functional as possible, and my code was much easier to test and debug later.
>
> So sure in C it makes less sense, but for me side effects are always bad, and they can actually
> be avoided or encapsulated in a lot of cases..
>
> By the way I created a target in the makefile and now every time I rebuild the slides
> they get synced with this public file, so actual slides are visible here:
>
> https://dl.dropbox.com/u/3183120/talks/unit_slides/unit.html
>
> for the side effect function I did a simple read from disk:
>
> def read_from_disk():
> """Read a file from disk and return a list with its content
> """
> assert path.isfile(FNAME), "File %s does not exist" % FNAME
> res = []
> for line in open(FNAME):
> res.append(line.strip())
>
> return res
>
>
> And show how I can test it increasing the smartness
>
>
> def test_read_from_disk():
> old_fname = with_side_one.FNAME
> with_side_one.FNAME = TMP
>
> open(TMP, 'w').write(TEXT)
> assert with_side_one.read_from_disk() == DESIRED
>
> remove(TMP)
> with_side_one.FNAME = old_fname
>
> # using unittest
>
> class TestReadFromDisk(unittest.TestCase):
> def setUp(self):
> self.old_fname = with_side_one.FNAME
> with_side_one.FNAME = TMP
> open(TMP, 'w').write(TEXT)
>
> def tearDown(self):
> with_side_one.FNAME = self.old_fname
> remove(TMP)
>
> def test_read_from_disk(self):
> res = with_side_one.read_from_disk()
> self.assertEqual(res, DESIRED)
>
>
> # using unittest and mock
> @patch('with_side_one.FNAME', new=TMP)
> class TestReadFromDiskMocked(unittest.TestCase):
> def setUp(self):
> open(TMP, 'w').write(TEXT)
>
> def tearDown(self):
> remove(TMP)
>
> def test_read_from_disk(self):
> res = with_side_one.read_from_disk()
> self.assertEqual(res, DESIRED)
>
>
>
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
More information about the testing-in-python
mailing list