python struct 从 Berkeley DB 解压一个特定的 C 结构

Question

I have the following C struct:

#define UUID4_LEN 37
...
typedef struct can_record {
    char id[UUID4_LEN];
    char *can_data;
} CAN_RECORD;

I am saving that record in Berkeley DB via the below function:

int insert_record(DB **dbpp, CAN_RECORD * record) {
    DB *db;
    DBT key, data;
    int ret;

    db = *dbpp;

    memset(&key, 0, sizeof(DBT));
    memset(&data, 0, sizeof(DBT));

    uuid4_generate(record->id);

    key.data = record->id;
    key.size = (u_int32_t)strlen(record->id) + 1;

    data.data = &record;
    data.size = sizeof(CAN_RECORD);

    ret = db->put(db, 0, &key, &data, 0);

    if(ret != 0) {
        fprintf(stderr, "Unable to insert record %s, err: %s\n", record->id,
            db_strerror(ret));

        return ret;
    }

    printf("Record inserted %s %s\n", record->id, record->can_data);

    return ret;
    
}

NOTE: the record->data has already been pre-populated previously, and it is of a variable length, but it is a stringified JSON structure, ie:

asprintf(&record.can_data, "{\"t\": \"%s\", \"b\": \"%s\", \"e\": \"%u\"\"}", U_UID, name, (unsigned)time(NULL));

I have a python process that reads the Berkeley DB (here is a small excerpt):

from berkeleydb import db
...
...
    cursor = self._db.cursor()
    record = cursor.first()
    while record:
        (id, data) = record
        self.log(f'RECORD: {id} {data}')

        id = struct.unpack("", id)
        data = struct.unpack("", data)

        self.log(f'DECODED: {id} {data}')

        record = cursor.next()
...

The record data looks like this:

b'46c54a16-366a-4397-aa68-357ab5538590\x00'

and

b'P\x99\x12\x00x\xbb\xfd~(\xbb\xfd~\x16\x00\x00\x00\x04\x00\x00\x00x\xbb\xfd~\x00\x00\x00\x00\x00\x00\x00\x00\xa4\x9f\x02A\x00\x00\x00\x00\x83.\xf0v@\x03\x00\x00\x08\x00\x00\x00\xf0h\x9e\x9fpo;\xcc\x1d\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00can0\x00\x00\x00\x00x\xbb\xfd~\x00\x00\x00\x00\x03\x00\x00\x00\xb0\xfa\x00A\x00\x00\x00\x00\x00\x00'

I am unable to figure out how I can use python's struct.unpack to decode the bytes string. I have tried variety of different formats, but have been unsuccessful.

How would you go about unpacking the struct, such that I have the original form.

Unfortunately, the Berkeley DB reader has to be in python.

Also note:

data.data = &record;
data.size = sizeof(CAN_RECORD);

the data is the entire struct, which includes the id[UUID4_LEN] , and the *can_data .

What would I need to do here:

    (id, data) = record
    id = struct.unpack("", id)
    data = struct.unpack("", data)

to achieve original form?

Answer 1

Ok, for the time being, I did a workaround. Rather than:

data.data = &record;
data.size = sizeof(CAN_RECORD);

I did:

data.data = record->can_data;
data.size = (u_int32_t)strlen(record->can_data) + 1;

So, I only saved the string, rather than the entire struct. Then, in my python, I simply did:

(id, data) = record        
id = str(id, 'utf-8')
data = str(data, 'utf-8')
                
self.log(f'ID: {id}') 
self.log(f'DATA: {data}')

and that decoded the byte string, to just a string perfectly:

ID: dacaf94f-ecf5-4252-89d8-e2c9deff8f8d
DATA: {"t": "", "b": "abc123", "e": "1653636766"}

Although, this has helped me progress without using python struct.unpack , I am still keen on understanding how to unpack structs in python, as I will likely have a future requirement with slightly more complicated struct definitions.