pandas how to avoid iterations on rows? applying function with params on each row

Question

using python 3.7+

want to split paragraphs into new rows using scispicy . need to use Spicy on each row to get the relevant result (not just split('.') ). Is it possible with pandas vectorization? any help would be much appreciated

have this df -

>>> df = pd.DataFrame({'num_legs': [2, 4, 8, 0],
...                    'num_wings': [2, 0, 0, 0],
...                    'some_description': ['falcons have wings. falcons fly', 'dog have 4 legs. they are the best', 'spiders create webs. spiders have 8 legs', 'fish swims. fish lives in water']},
...                    index=['falcon', 'dog', 'spider', 'fish'])
>>> df
        num_legs  num_wings  some_description
falcon         2          2   'falcons have wings. falcons fly'
dog            4          0   'dog have 4 legs. they are the best'
spider         8          0   'spiders create webs. spiders have 8 legs'
fish           0          0   'fish swims. fish lives in water'

have this function I want to apply on each row -

def split_to_sentences(text, nlp):
  doc = nlp(text)
  return list(doc.sents)

where nlp is nlp = spacy.load("en_core_sci_sm") I want to iterate over rows and split every sentence into 2 so the result would be -

        num_legs  num_wings  some_description
falcon         2          2   'falcons have wings.'
falcon         2          2   'falcons fly.'
dog            4          0   'dog have 4 legs'
dog            4          0   'they are the best'
spider         8          0   'spiders create webs'
spider         8          0   'spiders have 8 legs'
fish           0          0   'fish swims.'
fish           0          0   'fish lives in water'

maybe the only way is with iterrows/itertuples (which I understand are bad practice )?

edit 2 -

I understand apply is also a bad practice, is it possible to use something else?

Thank you

Answer 1

a = 'some_description'
df.assign(some_description=df[a].str.split(r'\. ')).explode(a)

result:

        num_legs    num_wings   some_description
falcon  2           2           falcons have wings
falcon  2           2           falcons fly
dog     4           0           dog have 4 legs
dog     4           0           they are the best
spider  8           0           spiders create webs
spider  8           0           spiders have 8 legs
fish    0           0           fish swims
fish    0           0           fish lives in water