How to delete blank lines from CSV file?

Question

I am trying to read a file using : as delimiter and contains blank lines. My CSV file looks likes-

P Data Usage (Bytes):        640
P Inst Usage (Bytes):        13322
P Local Patch Match:         105
P Global Patch Match:        20

V Data Usage (Bytes):        312
V Inst Usage (Bytes):        972
V Local Patch Match:         24
V Global Patch Match:        1

U UOP Usage:                 352
U Patch Match:               51

I want to remove blank lines from this file and I want to store it in the same file. I want to use all the values after : for updating Excel table.

How to remove blank lines from the file because if I am trying to read the file it is reading only top 4 lines and not reading any further line. I want to store every integer value in each row in a list like -

a = ['640', '13322', '105', '20', '312', '972', '24', ..., '51']

Answer 1

You don't have a CSV file, so don't try to treat it as one. CSV specifically means a file consisting of rows of columns, where all rows have the same number of columns and the columns are delimited by a single character (usually a comma or a tab).

There is no need to delete the lines from your source file if all you needed was those first 4 values for your code either.

Just loop over the lines in the file, split each line by whitespace , and if the resulting list is not empty, take the last value:

numbers = []
with open(filename) as inf:
    for line in inf:
        data = line.split()
        if data:
            numbers.append(data[-1])

str.split() with no arguments splits on arbitrary length whitespace , including tabs, spaces and newlines, with any whitespace at the start and end ignored.

When a line is empty the line string will be '\\n' , and splitting such a line results in an empty list. If there is anything other than whitespace on a line, you get a list of strings for each non-whitespace section:

>>> '\n'.split()
[]
>>> 'P Inst Usage (Bytes):        13322\n'.split()
['P', 'Inst', 'Usage', '(Bytes):', '13322']

Because your file has the numbers you want at the end of each line, taking the last element (with [-1] ) gets you all the numbers.

If you only need the top four such lines, test the len(numbers) result:

numbers = []
with open(filename) as inf:
    for line in inf:
        data = line.split()
        if data:
            numbers.append(data[-1])
            if len(numbers) == 4:
                break

The break stops the for loop, and when you exit the with statement, the file will be closed automatically.

If you do ever need to skip blank lines, just test if line.strip() is truthy; str.strip() uses the same arbitrary-width whitespace rules to remove whitespace from the start and end. Empty lines with just a newline separator end up as empty strings, which are considered false when used in a boolean test such as an if ...: condition:

if line.strip():
    # line is not empty

You could then write that to a new file if you so wish, but that's not really needed here .

Answer 2

Just strip the lines to check if they're empty.

with open('in.csv', 'r') as fpin, open('out.csv', 'w') as fpout:
    for line in fpin:
        if line.strip():
            fpout.write(line)

Answer 3

import pandas as pd
df = pd.read_csv('file.csv',sep=':')
df = df.dropna(how='all')