[TIP] Best practice for failing tests

James Cooke me at jamescooke.info
Fri Mar 23 03:35:50 PDT 2018


Thanks all for your really helpful suggestions!

I've gone for `xfail` with a `reason` and set `xfail_strict=true` in pytest.ini because it's helpful to be notified if the fake gets extended.

Cheers,

James




On Thu, 22 Mar 2018, at 7:39 PM, Chris Jerdonek wrote:
> On Thu, Mar 22, 2018 at 12:34 PM, Bruno Oliveira <nicoddemus at gmail.com> wrote:
> >> Another reason is flaky
> >> tests that sometimes fail and sometimes succeed, so that xfail()
> >> wouldn't work as a substitute.
> >
> >
> > Indeed, but in James' case I think `xfail(strict=True)` is the way to go,
> > because it will *fail* the test if it suddenly starts passing because the
> > fake now works as expected, which is a good thing because then that
> > opportunity can be used to remove the marker. A skip will stay in there
> > forever until someone takes notice by some other means.
> 
> Yes, I was going back and forth in my mind in my first reply to James,
> and xfail() does have that advantage. For example, if the developer
> maintaining the fake adds the functionality, you'd want to be notified
> when it occurs as opposed to having to remember to keep checking.
> 
> --Chris
> 
> 
> 
> >
> > Cheers,
> > Bruno.



More information about the testing-in-python mailing list