Using python shelve cross-platform

Question

I am hoping for a little advice on shelves/databases in Python.

Problem: I have a database created on the mac, that I want to use on windows 7. I use Python 3.2, MacOS 10.7, and win 7.

When I open and save my shelve on the mac all is good and well. I get a file with a ".db" extension. On my windows-python it is not recognized. I can however create a new db on the pc and get files with ".bak, dat, .dir" extensions.

I am guessing that the python on the pc does not have the same underlying database that my mac-python uses?

I am not sure which is the correct approach here, but maybe I could:

Change the default-db that my systems uses? Find out which db my mac-python uses and add that on the pc? Change the way I store my data all together?

Speed is not an issue, the datasize is a few megabytes, and it's not accessed very often.

Hope to find a helping hand out there. Thanks in advance - any help is much appreciated.

/Esben

What I am doing:

Import shelve
db = shelve.open('mydb')
entries = db['list']
db.close

It's pretty straight forward, I have a working db-file called "mydb.db" on the mac but when I try to open it on the pc-python i get:

Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/dbm/ init .py", line 107, in whichdb f = io.open(filename + ".pag", "rb") IOError: [Errno 2] No such file or directory: 'mydb.pag'

Answer 1

Thank you for your reply!

I seems that shelves in python are not easily forced to use a specific db, however pickles works like a charm. At least from mac os -> windows 7.

So short answer: If you want portability, don't use shelves, use pickles directly.

/Esben

Answer 2

sqlite3 module is a cross platform module that is even supported by many other languages and tools.

pickle module is simpler, but also cross platform. You give it an object and it dumps it to a file. No tables or rows like sqlite.

Answer 3

I ran into the same issue and implemented a dict-based class that supports loading and writing the contents of the dict from and to disk.

from pathlib import Path
import pickle


class DiskDict(dict):
    def __init__(self, sync_path: Path):
        self.path = sync_path

        if self.path.exists():
            with open(self.path, "rb") as file:
                tmp_dct = pickle.load(file)
                super().update(tmp_dct)
                print(f"loaded DiskDict with {len(tmp_dct)} items from {self.path}")

    def sync_to_disk(self):
        with open(self.path, "wb") as file:
            tmp_dct = super().copy()
            pickle.dump(tmp_dct, file)
            print(f"saved DiskDict with {len(tmp_dct)} items to {self.path}")

Answer 4

You might try using pysos , it's a lightweight shelve alternative that is also cross platform.

Install using pip: pip install pysos

Example usage:

import pysos

db = pysos.Dict('myfile.db')
db['hello'] = 'persistence!'
db.close()

The advantage is also that everything is contained in this single file myfile.db so you can easely just copy it around.