使用 pyreadstat 读取内存中的 SPSS 文件（.sav 或 .zsav）

Question

I've been developing a Django application. I know that there are some different of reading an SPSS file. One way is using pandas.

import pandas as pd

file_path = "./my_spss_file.sav"
df = pd.read_spss(file_path)

Another way is using pyreadstat

import pyreadstat
df, meta = pyreadstat.read_sav('./my_spss_file.sav')

As you can see above, unlike pandas, using using pyreadstat I can get the meta information such as variables and values of labels. So, that is what I am using. The problem with this pyreadstat is that I cannot use it for inmemory read. After uploading an spss file from a browser, each time I have to upload it to a directory and then read the file from there using pyreadstat module.

def upload_file(request):
    result = None
    # Get the context from the request.
    context = RequestContext(request)
    if request.is_ajax():
        if "POST" == request.method:
            global my_df
            global _explore
            global base_dir
            file = request.FILES['file']
            file_name = file.name
            base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
            try:
                my_df = None
                # Determine the type of the file and get the dataframe
                if file_name.endswith('.csv'):
                    my_df = pd.read_csv(file, header=0)
                elif file_name.endswith('.xlsx') or file_name.endswith('.xls'):
                    my_df = pd.read_excel(file, header=0)
                elif file_name.endswith('.sav') or file_name.endswith('.zsav'):
                    handle_uploaded_file(file, str(file))
                    file_path = os.path.join(base_dir, "upload\\") + file_name
                    my_df = util.read_spss_file(file_path)

def read_spss_file(f_name):
    df, meta = pyreadstat.read_sav(f_name, apply_value_formats=True)
    return df

def handle_uploaded_file(file, filename):
    upload_dir = os.path.join(base_dir, "upload\\") #base_dir + 'upload/'
    if not os.path.exists(upload_dir):
        os.mkdir(upload_dir)

    with open(upload_dir + filename, 'wb+') as destination:
        for chunk in file.chunks():
            destination.write(chunk)

I don't want to write an uploaded spss file to the disk. So, I was wondering whether there is a way to read an inmemory spss file using pyreadstat module?

Answer 1

Unfortunately it is not possible at the moment.

Pyreadstat relies on the C library Readstat which currently absolutely requires a file on disk.

The issue has been raised here .

Pandas read_spss also uses pyreadstat in the background, so both methods are actually the same.

Answer 2

class TempFile(type(pathlib.Path())):  # type: ignore
    def __exit__(self, exc_type, exc_val, exc_tb):
        filepath = str(self.absolute())
        try:
            os.remove(filepath)
        except OSError:
            logger.error('romve temporary file: %s failed!', filepath)
        self._closed = True

buffer = BytesIO()  # the bytes data
    
with TempFile('/tmp/file.sav') as fp:
try:
    fp.write_bytes(io.getvalue())
    return read_sav(fp, encoding=encoding)
except xxx:
    # do some fallback
    pass

this will help read from bytes in memeory