AttributeError: 'float' object has no attribute 'split' when making a generator object

Question

print([x["keywords"].split(",") for i,x in df.iterrows()  if not isinstance(x["keywords"], (int, float))])

print([x["tags"].split(",") for i,x in df.iterrows()  if not isinstance(x["tags"], (int, float))])

print([x["rating"].split(",") for i,x in df.iterrows()  if not isinstance(x["rating"], (int, float))])

print([x["rank"].split(",") for i,x in df.iterrows()  if not isinstance(x["rank"], (int, float))])

I want to join these four statements in a single statement when i concatenate them it gives me error:

AttributeError: 'float' object has no attribute 'split'

features = [(x["entity_id"], x["tags"].split(","),x["rating"],
           x["rank"],x["keywords"].split(",") )
           for (index, x) in df.iterrows() if not isinstance(x, (int, float))]

Answer 1

pd.DataFrame.iterrows returns tuples of index and pd.Series objects. Hence isinstance(x, (int, float)) isn't doing what you want it to, as a pd.Series object isn't a subclass of int or float . With this method you would need to iterate individual values contained within the pd.Series object.

This is possible, but I strongly advise against it. In fact, I advise you avoid iterrows altogether, as this loses all vectorised functionality, which is one of the main benefits of Pandas.

Here is a solution using pd.DataFrame.mask and NumPy arrays:

df = pd.DataFrame({'entity_id': ['SomeId', 3124123, 'SomeOtherId', 314324],
                   'tags': ['Tag1,Tag2', None, 'Tag4', 'Tag5,Tag6,Tag7'],
                   'rating': [5.0, 'SomeRating', 'SomeOtherRating', np.nan],
                   'rank': ['SomeRank', 2, np.nan, 4],
                   'keywords': ['key1', 'key2,key3', 'key4', 'key5']})

df2 = df.mask(df.apply(pd.to_numeric, errors='coerce').notnull() | df.isnull(), None)

for col in ['tags', 'keywords']:
    df2[col] = df2[col].str.split(',')

col_order = ['entity_id', 'tags', 'rating', 'rank', 'keywords']
res = [list(filter(None, x)) for x in df2[col_order].values.tolist()]

Result

print(res)

[['SomeId', ['Tag1', 'Tag2'], 'SomeRank', ['key1']],
 ['SomeRating', ['key2', 'key3']],
 ['SomeOtherId', ['Tag4'], 'SomeOtherRating', ['key4']],
 [['Tag5', 'Tag6', 'Tag7'], ['key5']]]

As a comment, this is pretty messy. It's good practice to decide on a consistent structure rather than this kind of mixed data type structure and filtering based on type.