Splitting sentences in a column and then appending in a data frame in python

Question

I have a data frame in python df.

Its structure is as follows :-

Sentences                 |    Value
This is my house           |      0
My house is good           |      2

. . . .

Now what I want it to split the column sentence to words and then have a pandas data frame to append these words with their original sentence value in front of them.

The output should be as follows:-

Words | Value
This  |   0
is    |   0
my    |   0
house |   0
My    |   2
house |   2
is    |   2
good  |   2  

. . .

I have used a function to split the sentences.

def makeTermsFrom(msg):
    return [m for m in msg.lower().split() if m]

a = readMessagesFromFile("./data/a_labelled.txt") #Returns a df
b = makeTermsFrom(a['Sentences'].iloc[0]) #Splits the sentences

but I was not able to add the words with their values in a df.

Answer 1

Use the DataFrame.itertuples() method:

import pandas as pd

df = pd.DataFrame(
    [['John Lennon', 10], ['George Harrison', 6]],
    columns=['beatle', 'songs']
)

longform = pd.DataFrame(columns=['word', 'num'])

for idx, name, songs in df.itertuples():
    name_words = (i.lower() for i in name.split())

    longform = longform.append(
        [{'word': nw, 'num': songs} for nw in name_words],
        ignore_index=True
    )

print(longform.head())

#        word  num
# 0      john   10
# 1    lennon   10
# 2    george    6
# 3  harrison    6