How do I convert binary data to hexadecimal through Python?

Question

So I'm trying to get the binary data field in a database as a hexadecimal string. I don't know if that is exactly what I'm looking for but that is my hunch. There is a column called status in my dataframe that says [binary data] in PostgreSQL but when I execute the following command it looks like this:

df = pd.read_sql_query("""SELECT * FROM public."Vehtek_SLG" LIMIT 1000""",con=engine.connect())

How do I get the actual data in that column?

Answer 1

It looks like your DataFrame has for each row a list of individual bytes instead of the entire hexadecimal bytes string. The Series df["status"].map(b"".join) will have the concatenated bytes strings.

import random

import pandas as pd


# Simulating lists of 10 bytes for each row
df = pd.DataFrame({
    "status": [
        [bytes([random.randint(0, 255)]) for _ in range(10)]
        for _ in range(5)
    ]
})

s = df["status"].map(b"".join)

Both objects look like:

# df
                                              status
0  [b'\xb3', b'f', b';', b'P', b'\xcb', b'\x9b', ...
1  [b'\xd2', b'\xe8', b'.', b'b', b'g', b'|', b'\...
2  [b'\xa7', b'\xe1', b'z', b'-', b'W', b'\xb8', ...
3  [b'\xc5', b'\xa9', b'\xd5', b'\xde', b'\x1d', ...
4  [b'\xa3', b'b', b')', b'\xe3', b'5', b'`', b'\...

# s
0       b'\xb3f;P\xcb\x9bi\xb0\x9e\xfd'
1            b'\xd2\xe8.bg|\x94O\x90\n'
2       b'\xa7\xe1z-W\xb8\xc2\x84\xb91'
3    b'\xc5\xa9\xd5\xde\x1d\x02*}I\x15'
4                b'\xa3b)\xe35`\x0ed#g'
Name: status, dtype: object

After coverting the status field to binary we can then use the following to make it hexadecimal.

df['status'] = s.apply(bytes.hex)

And now here is your field!

df['status'].head()

0    1f8b0800000000000400c554cd8ed33010beafb4ef6045...
1    1f8b0800000000000400c554cd8ed33010beafb4ef6045...
2    1f8b0800000000000400c554cd6e9b4010be47ca3bac50...
3    1f8b0800000000000400c554cd6e9b4010be47ca3bac50...
4    1f8b0800000000000400c554cd6e9b4010be47ca3bac50...