Python csv file can't read all rows

Question

There have total 36k rows in my csv file however it only processes 6k rows when I read the file. And it processes from bottom to up.

Here is my code

with open('ledgers.csv', 'r') as csv_ledger:
    r = csv.DictReader(csv_ledger)
    data = [dict(d) for d in r]

    groups = []
    count = 0

    for k, g in groupby(data, lambda r: (r['ref_num'])):
        groups.append({
            "ref_num": k,
            "items": [{k:v for k, v in d.items() if k not in ['ref_num']} for d in list(g)]
        })
        count += 1

print (json.dumps(groups,indent = 4))
print(count)

The count returned to me is 6199 only.

My file format as below

date       | ref_num | name        <---Header
2019-01-01 | H001    | shanny

Answer 1

If I'm reading it right, it looks like you are only finding 6000 unique reference numbers. You may be fetching more than 6000 rows, but you are getting 6000 groups.

To get a proper count on the number of rows read from your csv file, try using

print(len(data))

Instead of

print(count)

Because count only indicates the number of unique reference number groups you have.