Decode base64 array at specific index positions

Question

I handle a big number of XML files, each of which stores a float64 array in the form of a base64 string. For each of these files, I would like to access the value of its array at a specific index without loading the whole array. I wish to do this in Python.

Currently, I decode the entire string and then access the value at given index, but I am looking for a much faster method.

Answer 1

As I understand, each float uses 8 bytes to store. Each 3-bytes of binary data are mapped to 4 base64 chars. So, in order to access i-th float, you'll need to:

• get base64 chars from index (i * 8 / 3) * 4 to (i * 8) / 3) * 4 + 4
• decode that chunk from base64 to binary
• drop first (i * 8) % 3 bytes
• use next 8 bytes as a float64 value.