[TIP] Multiplatform testing

[Manuel de la Pena Saenz]

Hello *,

At the moment I'm working on the port to Windows of some code that was initially written to work only on Linux (desktopcouch). To make the port easier, I'm currently working on ensuring that the tests that were written can be ran on Windows. This has lead me to a problem I really do not know how to solve:

Desktopcouch uses dbus to advertise the port in which the users personal CouchDb is running. A certain amount of our tests are integration tests that require dbus to be running. As you can imaging this tests fail on Windows but should not be considered to be a fail since it is "an expected fail". To load an run or test I use u1trial, which can be found in Launchpad (launchpad.net/ubuntuone-dev-tools) to which I am adding an @skip decorator (and similar) to work with twisted.trial.unittest.TestCase so that we can skip tests (the code can be found at at lp:~mandel/ubuntuone-dev-tools/load_test_according_to_platform in testcase.py). The @skipIf approach works on those tests that do not have imports which cannot be found on Windows (for example, it does not have import dbus), however if the imports are present when loading the test we have an import error and we never get the SkipTest exception. This ImportErrors of course occur due to the fact that u1trial  uses __import__ to load the test cases and the module to load has an "platform illegal import"

My first attempt to solve this situation has been to write a decorator that will skip a test case if one of its imports fails and if they do not fail, it will add the modules to the func_globals of the test. The code of such an attempt is the following:

def skipIfNotModules(modules, reason):
    """
    Skip tests when a set of modules is missing.
    """
    if modules:
        def decorator(test_item):
            if not (isinstance(test_item, type) and\
                issubclass(test_item, TestCase)):
                def decorator(test_item):
                    @wraps(test_item)
                    def import_wrapper(*args, **kwargs):
                        for module_to_import in modules:
                            try:
                                test_item.func_globals[module_to_import] = __import__(module_to_import)
                            except ImportError:
                                raise SkipTest(reason)
            		return test_item(*args, **kwargs)
            		test_item = import_wrapper
            		return test_item
            	return decorator
            # tell twisted.trial.unittest to skip the test, pylint will 
            # complain since it thinks we are redefining a name out 
            # of the scope
            # pylint: disable=W0621,W0612
            test_item.skip = reason
            # pylint: enable=W0621,W0612
            # because the item was skipped, we will make sure that no
            # services are started for it
            if hasattr(test_item, "required_services"):
                # pylint: disable=W0612
                test_item.required_services = lambda *args, **kwargs: []
                # pylint: enable=W0612
            return test_item
        return decorator
    return _id
While the above code works for a test method, it does not for a TestCase. I was wondering if someone in the list has experience with this type of situation in which tests have to be skipped when the are loaded by a test runner like u1trial depending on the platform. I am specially interested in those examples where the information of which test to be run does not have to be supplied to the test loader in an "artificial way" (I consider to be an artificial way to pass a dict with the os.platform as the keys and the value a list of the test to run).

I would greatly appreciate any input in this particular problem as well as to the more general question of, how do I plan my tests so that we can work in a multi-platform env?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20110103/b2e78d0c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20110103/b2e78d0c/attachment.pgp>