[pygr-notify] Issue 52 in pygr: Move from shelve to sqlite3 storage?

[codesite-noreply at google.com]

Status: Accepted
Owner: cjlee112
CC: ti... at idyll.org,  deepreds,  istvan.albert
Labels: Type-Defect Priority-Medium

New issue 52 by cjlee112: Move from shelve to sqlite3 storage?
http://code.google.com/p/pygr/issues/detail?id=52

shelve has caused us problems over and over, due to incompatible formats,
poor scalability etc.  Now Python 2.6 and 3 have completely broken shelve
support, by removing bsddb from the standard library.  That means shelve
will no longer be able to read data from earlier versions of Python, where
shelve uses bsddb storage.  It seems like the only way to end the pain is
to stop relying on shelve.

On the positive side, sqlite3 is now included in the standard library (as
of python 2.5).  Hopefully it provides decent scalability, in addition to
the much greater capabilities of an SQL database.  Ideally, this would
permit us to standardize most storage as SQL databases, accessed using our
standard classes.  This could simplify a lot of code.  This also suggests
that our code wouldn't care where the storage is: in an sqlite3 file vs. in
a MySQL server or some other relational database.  That opens up a lot of
possibilities for moving data around flexibly.

Questions:
- is sqlite3 genuinely scalable??  Someone should test the hell out of it.
  The poor scalability of default shelve came as a nasty surprise, which I
wouldn't like to repeat on another Standard Library storage component.

- this raises annoying platform issues, since sqlite3 only comes standard
in Python 2.5 and later.  A lot of people still use 2.3 or 2.4...  Is it
easy to install pysqlite / sqlite3 on those platforms?

- of course, we're stuck with providing backwards compatibility with shelve
data.  Presumably that means we retain our old capabilities, but only add
new capabilities to the SQL based components: i.e. old data, older feature
set; new data, new & improved feature set.

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings