数据框fit_transform抛出错误，看似错误

Question

I am running the given line in Python:

df = df.apply(lambda x: d[x.name].fit_transform(x))

And getting the following error:

~/anaconda3/envs/python3/lib/python3.6/site-packages/numpy/lib/arraysetops.py in _unique1d(ar, return_index, return_inverse, return_counts)
278 
279     if optional_indices:
--> 280         perm = ar.argsort(kind='mergesort' if return_index else 'quicksort')
281         aux = ar[perm]
282     else:

TypeError: ("'<' not supported between instances of 'str' and 'float'", 'occurred at index name')

I don't have the character '<' anywhere in my file, so not sure what the error is?

New to Python, so any insights on how to understand these errors, greatly appreciated.

Answer 1

I think this may be happening because you are not passing clean or correct data to your fit_transform . It's hard to tell without an answer to my question in the comments (What does the d stand for in df = df.apply(lambda x: d[x.name].fit_transform(x)) ?)

I took some dummy data and made an example of how you might apply a fit_transform to a dataframe with apply.

import random
import pandas as pd
import numpy as np

# Random dummy data
s = "Crime Type Summer|Crime Type Winter".split("|")
j = {x: [random.choice(["ASB", "Violence", "Theft", "Public Order", "Drugs"]) for j in range(300)] for x in s}
df = pd.DataFrame(j)

# Instantiate the vectorizer for use in the lambda function.
from sklearn.feature_extraction.text import CountVectorizer
cv = CountVectorizer()

# Now we can call the transform directly in the lambda function.
df = df.apply(lambda x: cv.fit_transform(df[x.name].values))

This completes successfully and gives:

Crime Type Summer      (0, 1)\t1\n  (1, 4)\t1\n  (2, 2)\t1\n  (2, 3...
Crime Type Winter      (0, 5)\t1\n  (1, 0)\t1\n  (2, 0)\t1\n  (3, 5...
dtype: object