[TIP] Multiplatform testing

[Michael Foord]

On 03/01/2011 17:13, Manuel de la Pena Saenz wrote:
> Hello *,
>
> At the moment I'm working on the port to Windows of some code that was 
> initially written to work only on Linux (desktopcouch). To make the 
> port easier, I'm currently working on ensuring that the tests that 
> were written can be ran on Windows. This has lead me to a problem I 
> really do not know how to solve:
>
> Desktopcouch uses dbus to advertise the port in which the users 
> personal CouchDb is running. A certain amount of our tests are 
> integration tests that require dbus to be running. As you can imaging 
> this tests fail on Windows but should not be considered to be a fail 
> since it is "an expected fail". To load an run or test I use u1trial, 
> which can be found in Launchpad (launchpad.net/ubuntuone-dev-tools 
> <http://launchpad.net/ubuntuone-dev-tools>) to which I am adding an 
> @skip decorator (and similar) to work with 
> twisted.trial.unittest.TestCase so that we can skip tests (the code 
> can be found at 
> at lp:~mandel/ubuntuone-dev-tools/load_test_according_to_platform in 
> testcase.py). The @skipIf approach works on those tests that do not 
> have imports which cannot be found on Windows (for example, it does 
> not have import dbus), however if the imports are present when loading 
> the test we have an import error and we never get the SkipTest 
> exception. This ImportErrors of course occur due to the fact that 
> u1trial  uses __import__ to load the test cases and the module to load 
> has an "platform illegal import"
>

I solve this problem like this:

import unittest2
try:
     import module
except ImportError:
     module = None

class SomeTest(unittest2.TestCase):
     @unittest2.skipIf(module is None, "Test requires module")
     def test_something(self):
         ...

The skip decorators can be applied to a whole class as well as methods 
from Python 2.6.

An alternative approach:

try:
   import  somemodule
except  ImportError:
   somemodule  =  None


def error_if_missing(name):
   failed = False
   try:
     __import__(name)
   except:
     failed = True
   def  decorator(f):
      def  inner(self):
        if  failed:
          raise  ImportError('name')
        return  f(self)
      return  inner
   return decorator

class  SomeTest(unittest2.TestCase):

   @error_if_missing('somemodule')
   def  test_something2(self):
     ...

I dislike the "auto-adding to module globals" approach as it means a core part of the setup is done behind the scenes and invisible.

All the best,

Michael Foord


> My first attempt to solve this situation has been to write a decorator 
> that will skip a test case if one of its imports fails and if they do 
> not fail, it will add the modules to the func_globals of the test. The 
> code of such an attempt is the following:
>
> def  skipIfNotModules(modules,  reason):
>      """
>      Skip tests when a set of modules is missing.
>      """
>      if  modules:
>          def  decorator(test_item):
>              if  not  (isinstance(test_item,  type)  and\
>                  issubclass(test_item,  TestCase)):
>                  def  decorator(test_item):
>                      @wraps(test_item)
>                      def  import_wrapper(*args,  **kwargs):
>                          for  module_to_import  in  modules:
>                              try:
>                                  test_item.func_globals[module_to_import]  =  __import__(module_to_import)
>                              except  ImportError:
>                                  raise  SkipTest(reason)
>              		return  test_item(*args,  **kwargs)
>              		test_item  =  import_wrapper
>              		return  test_item
>              	return  decorator
>              # tell twisted.trial.unittest to skip the test, pylint will
>              # complain since it thinks we are redefining a name out
>              # of the scope
>              # pylint: disable=W0621,W0612
>              test_item.skip  =  reason
>              # pylint: enable=W0621,W0612
>              # because the item was skipped, we will make sure that no
>              # services are started for it
>              if  hasattr(test_item,  "required_services"):
>                  # pylint: disable=W0612
>                  test_item.required_services  =  lambda  *args,  **kwargs:  []
>                  # pylint: enable=W0612
>              return  test_item
>          return  decorator
>      return  _id
> While the above code works for a test method, it does not for a 
> TestCase. I was wondering if someone in the list has experience with 
> this type of situation in which tests have to be skipped when the are 
> loaded by a test runner like u1trial depending on the platform. I am 
> specially interested in those examples where the information of which 
> test to be run does not have to be supplied to the test loader in an 
> "artificial way" (I consider to be an artificial way to pass a dict 
> with the os.platform as the keys and the value a list of the test to run).
>
> I would greatly appreciate any input in this particular problem as 
> well as to the more general question of, how do I plan my tests so 
> that we can work in a multi-platform env?
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20110103/23562276/attachment-0001.htm>