[TIP] Multiplatform testing

Manuel de la Pena Saenz manuel.delapena at canonical.com
Mon Jan 3 13:28:57 PST 2011

On 03 Jan 2011, at 18:34, Michael Foord wrote:

> On 03/01/2011 17:13, Manuel de la Pena Saenz wrote:
>> Hello *,
>> At the moment I'm working on the port to Windows of some code that was initially written to work only on Linux (desktopcouch). To make the port easier, I'm currently working on ensuring that the tests that were written can be ran on Windows. This has lead me to a problem I really do not know how to solve:
>> Desktopcouch uses dbus to advertise the port in which the users personal CouchDb is running. A certain amount of our tests are integration tests that require dbus to be running. As you can imaging this tests fail on Windows but should not be considered to be a fail since it is "an expected fail". To load an run or test I use u1trial, which can be found in Launchpad (launchpad.net/ubuntuone-dev-tools) to which I am adding an @skip decorator (and similar) to work with twisted.trial.unittest.TestCase so that we can skip tests (the code can be found at at lp:~mandel/ubuntuone-dev-tools/load_test_according_to_platform in testcase.py). The @skipIf approach works on those tests that do not have imports which cannot be found on Windows (for example, it does not have import dbus), however if the imports are present when loading the test we have an import error and we never get the SkipTest exception. This ImportErrors of course occur due to the fact that u1trial  uses __import__ to load the test cases and the module to load has an "platform illegal import"
> I solve this problem like this:
> import unittest2
> try:
>     import module
> except ImportError:
>     module = None
> class SomeTest(unittest2.TestCase):
>     @unittest2.skipIf(module is None, "Test requires module")
>     def test_something(self):
>         ...

Beautiful, I cannot believe I did not think about that! 

> The skip decorators can be applied to a whole class as well as methods from Python 2.6.

I knew that, I had to implement my own for twisted.trial.unittest following the implementation in unnittest2 and works great :)

> An alternative approach:
> try:
>   import somemodule
> except ImportError:
>   somemodule = None
> def error_if_missing(name):
>   failed = False
>   try:
>     __import__(name)
>   except:
>     failed = True
>   def decorator(f):
>     def inner(self):
>       if failed:
>         raise ImportError('name')
>       return f(self)
>     return inner
>   return decorator
> class SomeTest(unittest2.TestCase):
>   @error_if_missing('somemodule')
>   def test_something2(self):
>     ...
> I dislike the "auto-adding to module globals" approach as it means a core part of the setup is done behind the scenes and invisible.
> All the best,

Thanks a lot!

> Michael Foord
>> My first attempt to solve this situation has been to write a decorator that will skip a test case if one of its imports fails and if they do not fail, it will add the modules to the func_globals of the test. The code of such an attempt is the following:
>>  def skipIfNotModules(modules, reason):
>>     """
>>     Skip tests when a set of modules is missing.
>>     """
>>     if modules:
>>         def decorator(test_item):
>>             if not (isinstance(test_item, type) and\
>>                 issubclass(test_item, TestCase)):
>>                 def decorator(test_item):
>>                     @wraps(test_item)
>>                     def import_wrapper(*args, **kwargs):
>>                         for module_to_import in modules:
>>                             try:
>>                                 test_item.func_globals[module_to_import] = __import__(module_to_import)
>>                             except ImportError:
>>                                 raise SkipTest(reason)
>>             		return test_item(*args, **kwargs)
>>             		test_item = import_wrapper
>>             		return test_item
>>             	return decorator
>>             # tell twisted.trial.unittest to skip the test, pylint will 
>>             # complain since it thinks we are redefining a name out 
>>             # of the scope
>>             # pylint: disable=W0621,W0612
>>             test_item.skip = reason
>>             # pylint: enable=W0621,W0612
>>             # because the item was skipped, we will make sure that no
>>             # services are started for it
>>             if hasattr(test_item, "required_services"):
>>                 # pylint: disable=W0612
>>                 test_item.required_services = lambda *args, **kwargs: []
>>                 # pylint: enable=W0612
>>             return test_item
>>         return decorator
>>     return _id
>> While the above code works for a test method, it does not for a TestCase. I was wondering if someone in the list has experience with this type of situation in which tests have to be skipped when the are loaded by a test runner like u1trial depending on the platform. I am specially interested in those examples where the information of which test to be run does not have to be supplied to the test loader in an "artificial way" (I consider to be an artificial way to pass a dict with the os.platform as the keys and the value a list of the test to run).
>> I would greatly appreciate any input in this particular problem as well as to the more general question of, how do I plan my tests so that we can work in a multi-platform env?
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
> -- 
> http://www.voidspace.org.uk/
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing http://www.sqlite.org/different.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20110103/591b14e0/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20110103/591b14e0/attachment.pgp>

More information about the testing-in-python mailing list