[TIP] virtual filesystem

[Michael Foord]

On 06/12/2011 14:19, Michael Foord wrote:
> On 06/12/2011 14:07, exarkun at twistedmatrix.com wrote:
>> On 10:20 am, andrea.crotti.0 at gmail.com wrote:
>>> I have a lot of code which has to manipulate and handle the 
>>> filesystem, creating
>>> directories and so on.
>>>
>>> The testing now is far from complete, because even if I tried to use 
>>> a functional
>>> paradigm, still many things are too bound to the "real world".
>>> And it's really time for me to learn how to use mock objects ;)
>>>
>>> Before I waste too much time trying (and failing a few times), any 
>>> suggestions of how
>>> it could be possible?
>>>
>>>
>>> I thought for example something like:
>>> class FileSystemMock(object):
>>>     """Create a filesystem like object
>>>     """
>>>     def __init__(self, initial_state):
>>>         self.state = dict(initial_state) if initial_state else {}
>>>
>>>     # what are the various things
>>>
>>>
>>> Where I have an initial state of the directory structure, and every 
>>> operation
>>> will manage this dictionary of directories, raising the right errors 
>>> when needed.
>>>
>>> The problem is that I should rewrite the whole "os." if I want to do 
>>> something
>>> general, or is there a better way?
>>
>> A while ago I started working on something like this.  Since `os` is 
>> such a fundamental part of I/O in Python, I think it's necessary to 
>> fake it.  From there you can proceed to something more general.
>>
>> However, it's true that `os` is a sprawling mess, and it proved to be 
>> more effort than I cared to invest.  Hence my work is incomplete.  
>> You can see it here:
>>
>>   http://twistedmatrix.com:12435/2931
>>
>> in case you're curious.  If you wanted to pick it up, that'd be fine 
>> with me. ;)
>
> A full mock filesystem that can patch and unpatch os / os.path 
> functions sounds like a great tool.
>
> I wrote a mock "open" that implements the full file api and uses a 
> configurable backend, for "Try Python" [1].
>
>     
> http://code.google.com/p/trypython/source/browse/trunk/trypython/app/storage.py
>


Hmm... for some value of "full file api". It was implemented for Python 
2.5 compatibility, with the following notes in the tests:

Differences from standard file type:

* Attempting to set the read-only attributes (like mode, name etc) 
raises an AttributeError
   rather than a TypeError
* The exception messages are not all identical (some are better!)
* Strict about modes. Unrecognised modes always raise an exception

   (NOTE: The exception method that the standard file type does throw is:
    "ValueError: mode string must begin with one of 'r', 'w', 'a' or 
'U', not 'z'")

* The deprecated readinto is not implemented

TODO:

* The buffering argument to the constructor is not implemented
* The IOError exceptions raised don't have an associated errno
* encoding, errors and newlines do nothing
* Behavior of tell() and seek() for text mode files may be incorrect (it
   should treat '\n' as '\r\n')
* Behaves like Windows, writes '\n' as '\r\n' unless in binary mode. A 
global
   flag to control this?
* Universal modes not supported
* Missing __format__ method needed when we move to 2.6
* Implementations of os and os.path that work with storage_backend

You're probably better off with Jean-Paul's... :-)

All the best,

Michael

> It uses functions in a "storage_backend" module to do that actual 
> reading or writing. (In the case of Try Python it was using browser 
> storage). You can write a custom backend by implementing four functions:
>
>     
> http://code.google.com/p/trypython/source/browse/trunk/trypython/app/storage_backend.py
>
> Tests here:
>
>     
> http://code.google.com/p/trypython/source/browse/trunk/trypython/app/tests/test_storage.py
>
> I always intended to, but never got round to writing os / os.path 
> functions...
>
> All the best,
>
> Michael Foord
>
>
>
> [1] http://trypython.org (uses Silverlight)
>
>
>>
>> Jean-Paul
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>>
>
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html