如何遍历多个numpy数组并将项目从一个数组追加到另一个具有相同ID的数组？

Question

What if I have the following data, test_df['review_id'] that contains the id of the dataframe. I need to pair each of them with data from other arrays. I am going to have a code like the following.

def classify_nb_report(X_train_vectorized, y_train, X_test_vectorized, y_test):
    clf = MultinomialNB()

    # TRAIN THE CLASSIFIER WITH AVAILABLE TRAINING DATA
    clf.fit(X_train_vectorized, y_train)

    y_pred_class = clf.predict(X_test_vectorized)

    return y_pred_class

for i in range(0, n_loop):
    train_df, test_df = train_test_split(df, test_size=0.3)
    ....
    nb_y = classify_nb_report(X_train_vectorized, y_train, X_test_vectorized, y_test)

As you can see above, in each iteration I am going to get a new set of nb_y which is a numpy array. I am also going to have different sets of test_df and train_df (which are randomly chosen by the function above). I want to pair each value of nb_y from each iteration to id that matches test_df['review_id'] .

With the following code, I can get the id of test_df side by side with the value from nb_y .

for f, b in zip(test_df['review_id'], nb_y):
    print(f, b)

Result:

17377 5.0
18505 5.0
24825 1.0
16032 5.0
23721 1.0
18008 5.0

Now, what I want is, from the result above, I append the values of nb_y from the next iterations to their corresponding ids.

I hope this is not too confusing, I will try to expand more if my question is not clear enough. Thanks in advance.

Answer 1

I am not sure if I understand the problem correctly and how the rest of your code works but I assume the following code might do what you need. Let me know if it works or if there is something wrong with the answer.

dictionary = {}
for i in range(0, n_loop):
    train_df, test_df = train_test_split(df, test_size=0.3)
    ....
    nb_y = classify_nb_report(X_train_vectorized, y_train, X_test_vectorized, y_test)
    id = test_df['review_id']
    if not id in dictionary.keys():
        dictionary[id] = [nb_y]
    else:
        dictionary[id].append(nb_y)

Answer 2

After referring to this and this , I finally came up with my own solution. I turned the code above into something like this.

def classify_nb_report(X_train_vectorized, y_train, X_test_vectorized, y_test):
    clf = MultinomialNB()

    # TRAIN THE CLASSIFIER WITH AVAILABLE TRAINING DATA
    clf.fit(X_train_vectorized, y_train)

    y_pred_class = clf.predict(X_test_vectorized)

    return y_pred_class


nb_y_list = []

for i in range(0, n_loop):
    train_df, test_df = train_test_split(df, test_size=0.3)
    ....
    nb_y = classify_nb_report(X_train_vectorized, y_train, X_test_vectorized, y_test)

    nb_y_list.extend([list(x) for x in zip(test_df['review_id'],nb_y)])

dd = defaultdict(list)
for key, val in nb_y_list:
     dd[key].append(val)
     print(dd)

Basically, I made an empty list called nb_y_list first. Then for each iteration, I zip the id from test_df['review_id'] to be parallel with the value from nb_y , and extend them to the previous nb_y_list . After all the loops are finished, I will get the complete list that I now I will need to convert to dictionary using defaultdict() .