Python Flask - How to Convert Empty Strings to None type in list of lists

Question

I have a list of lists that holds the values from reading in a csv file. My problem is, my empty cells are getting read in as '' , rather than None . I need to insert these values into the database, so I have to switch all the empty strings to None in order for the empty values to show as NULL in the database.

My question is, how do I convert empty strings to None in a list of lists.

I have tried this conversion using lambda, and it works, however, only for a flattened list (one big list), but not a list of lists . This was the implementation:

    first_four = [sublist[0:4] for sublist in allocations]

    # Converting empty cells to None
    flat_ls = [item for sublist in first_four for item in sublist]
    conv = lambda i: i or None
    res = [conv(i) for i in flat_ls]

In essence, the above code has this output:

Example flattened list: ['a', '', 'c', '', '', f, g, h, i]
The list after conversion of Empty Strings: ['a', None, 'c', None, None, f, g, h, i]

But I need this implementation for a list of lists as such:

Example list of lists: [['a', '', 'c'],['', '', 'f'],['g', 'h', 'i']]
The list of lists after conversion of Empty Strings: [['a', None, 'c'],[None, None, 'f'],['g', 'h', 'i']]

Any suggestions how I could implement this? TIA!

Answer 1

First use (two) normal for -loops

data = [['a', '', 'c'],['', '', 'f'],['g', 'h', 'i']]

new_data = []

for row in data:
    new_row = []
    for cell in row:
        if cell == '':
            cell = None
        new_row.append(cell)
    new_data.append(new_row)

print(new_data)

Result:

[['a', None, 'c'], [None, None, 'f'], ['g', 'h', 'i']]

And later you can try to convert it to list comprehension.

First step - convert inner for -loop into list comprehension

data = [['a', '', 'c'],['', '', 'f'],['g', 'h', 'i']]

new_data = []

for row in data:
    new_row = [None if cell == '' else cell for cell in row]
    new_data.append(new_row)

print(new_data)

Second step - convert outer for -loop into list comprehension

data = [['a', '', 'c'],['', '', 'f'],['g', 'h', 'i']]

new_data = [
    [None if cell == '' else cell for cell in row]
    for row in data
]

print(new_data)

---

EDIT:

You may also add your i or None but without lambda

data = [['a', '', 'c'],['', '', 'f'],['g', 'h', 'i']]

new_data = [
    [cell or None for cell in row]
    for row in data
]

print(new_data)

And you can write it in one line

new_data = [ [cell or None for cell in row] for row in data ]

and eventually you can use shorter names

new_data = [ [c or None for c in r] for r in data ]

---

EDIT:

With numpy.array you could do it simpler arr[ arr == '' ] = None

data = [['a', '', 'c'],['', '', 'f'],['g', 'h', 'i']]

import numpy as np

# convert list to array
arr = np.array(data, dtype=object)

# replace
arr[ arr == '' ] = None

# convert array to list
new_data = arr.tolist()

Something similar you could do with pandas.DataFrame