在迭代时从 pandas dataframe 中删除行

Question

I have the following python script. In it, I am iterating through a CSV file which has rows and rows of loyalty cards. In many cases, there is more than one entry per card. I am currently looping through each row, then using loc to find all other instances of the card in the current row, so I can combine them together to post to an API. What I'd like to do however, is when that post is done, remove all the rows I've just merged, so that way the iteration doesn't hit them again.

That's the part I'm stuck on. Any ideas? Essentially I want to remove all the rows in card_list from csv before I go for the next iteration. That way even though there might be 5 rows with the same card number, I only process that card once. I tried by using

csv = csv[csv.card != row.card]

At the end of the loop, thinking it might re-generate the dataframe without any rows with a card matching the one just processed, but it didn't work.

import urllib3
import json
import pandas as pd 
import os
import time 
import pyfiglet
from datetime import datetime
import array as arr

    for row in csv.itertuples():
        dt = datetime.now()
        vouchers = []
        if minutePassed(time.gmtime(lastrun)[4]):
            print('Getting new token...')
            token = get_new_token()
            lastrun = time.time()
        print('processing ' + str(int(row.card)))
        card_list = csv.loc[csv['card'] == int(row.card)]
        print('found ' + str(len(card_list)) + ' vouchers against this card')

        for row in card_list.itertuples():
            print('appending card ' + str(int(row.card)) + ' voucher ' + str(row.voucher))
            vouchers.append(row.voucher)
        print('vouchers, ', vouchers)

        encoded_data = json.dumps({
            "store_id":row.store,
            "transaction_id":"11111",
            "card_number":int(row.card),
            "voucher_instance_ids":vouchers
        })
        print(encoded_data)
        number += 1

        r = http.request('POST', lcs_base_path + 'customer/auth/redeem-commit',body=encoded_data,headers={'x-api-key': api_key, 'Authorization': 'Bearer ' + token})
        response_data = json.loads(r.data)

        if (r.status == 200):
            print (str(dt) + ' ' + str(number) + ' done. processing card:' + str(int(row.card)) + ' voucher:' + str(row.voucher) + ' store:' + str(row.store) + ' status: ' + response_data['response_message'] + ' request:' + response_data['lcs_request_id'])
        else:
            print (str(dt) + ' ' + str(number) +  'done. failed to commit ' + str(int(row.card)) + ' voucher:' + str(row.voucher) + ' store:' + str(row.store) + ' status: ' + response_data['message'])
            new_row = {'card':row.card, 'voucher':row.voucher, 'store':row.store, 'error':response_data['message']}
            failed_csv = failed_csv.append(new_row, ignore_index=True)
            failed_csv.to_csv(failed_csv_file, index=False)
            csv = csv[csv.card != row.card]
    print ('script completed')
    print (str(len(failed_csv)) + ' failed vouchers will be saved to failed_commits.csv')
    print("--- %s seconds ---" % (time.time() - start_time))

Answer 1

First rule of thumb is never alternate what you are iterating on . Also, I think you are doing it wrong with itertuples . Let's do groupby:

for card, card_list in csv.groupby('card'):
    # card_list now contains all the rows that have a specific cards
    # exactly like `card_list` in your code
    print('processing, card)
    print('found', len(card_list), 'vouchers against this card')

    # again `itertuples` is over killed -- REMOVE IT
    # for row in card_list.itertuples():

    encoded_data = json.dumps({
            "store_id": card_list['store'].iloc[0],      # same as `row.store`
            "transaction_id":"11111",
            "card_number":int(card),
            "voucher_instance_ids": list(card_list['voucher']) # same as `vouchers`
        })
    
    # ... Other codes