Create pandas df from raw text file

Question

I have a text file that I would like to be formatted into a pandas dataframe. It is read as a string in the form of:
print(text)=

product: 1
description: product 1 desc
rating: 7.8
review: product 1 review

product: 2
description: product 2 desc
rating: 4.5
review: product 2 review

product: 3
description: product 3 desc
rating: 8.5
review: product 3 review

I figured I would split them by using text.split('\n\n') to group them into lists. I would assume iterating each into a dict, then loading to a pandas df would be a good route, but I am having trouble doing so. Is this the best route, and could someone please help me get this into a pandas df?

Answer 1

You can use read_csv with create groups by compare first column by product string and pivot :

df = pd.read_csv('file.txt', header=None, sep=': ', engine='python')
df = df.assign(g = df[0].eq('product').cumsum()).pivot('g',0,1)
print (df)
0      description product rating             review
g                                                   
1   product 1 desc       1    7.8   product 1 review
2   product 2 desc       2    4.5   product 2 review
3   product 3 desc       3    8.5   product 3 review

Or create list of dictionaries:

#https://stackoverflow.com/a/18970794/2901002
data = []
current = {}
with open('file.txt') as f:
    for line in f:
        pair = line.split(':', 1)
        if len(pair) == 2:
            if pair[0] == 'product' and current:
                # start of a new block
                data.append(current)
                current = {}
            current[pair[0]] = pair[1].strip()
    if current:
        data.append(current)
        
df = pd.DataFrame(data)
print (df)
  product     description rating            review
0       1  product 1 desc    7.8  product 1 review
1       2  product 2 desc    4.5  product 2 review
2       3  product 3 desc    8.5  product 3 review

Or reshape each 4 values to 2d numpy array and pass to DataFrame constructor:

df = pd.read_csv('file.txt', header=None, sep=': ', engine='python')

df = pd.DataFrame(df[1].to_numpy().reshape(-1, 4), columns=df[0].iloc[:4].tolist())
print (df)
  product     description rating            review
0       1  product 1 desc    7.8  product 1 review
1       2  product 2 desc    4.5  product 2 review
2       3  product 3 desc    8.5  product 3 review