Python double list comprehension

Question

I have a list of strings read from a file I need to turn into a 2D array:

IN >> lines = ['0.1;0;0;0;', '0.2;0;0;0;', '0.3;1;1;1;', '0.4;2;2;2;', '0.5;0;0;0;']
# Goes on for a few thousand lines

Note each string ends in ; .

I need to exclude lines with all zeroes (some lines are all zeroes at the start and at the end)

I .split() on ; but filter None to remove the empty values I will get at the end of each returned array from the .split() .

data_array = [list(filter(None, line.split(';'))) for line in lines if line.split(';')[1] != '0']

OUT >> data_array = [[0.1, 0, 0, 0], [0.2, 0, 0, 0], [0.3, 1, 1, 1], [0.4, 2, 2, 2], [0.5, 0, 0, 0]]

This is kind of returning the 2D array I need, but including the arrays with all zeroes, so my conditional must be at the wrong place. Except I thought conditions at the end of a list comprehension filter the elements that go into the list.

Then I thought I needed to filter on the "inside" array:

data_array = [[l for l in (filter(None, line.split(';'))) if l != '0'] for line in lines]

OUT >> data_array = [[0.1], [0.2], [0.3, 1, 1, 1], [0.4, 2, 2, 2], [0.5]]

Except this is removing the zeroes but leaving the marker (the first element of each array is a marker)

What I'm looking to get is just the arrays that contain numbers but not zeroes

DESIRED OUTPUT >> data_array = [[0.3, 1, 1, 1], [0.4, 2, 2, 2]]

Answer 1

It is cleaner to do this over multiple lines and not using the filter(None, ...) as you can just use line[:-1] , to emit the last character. First creating the list in lists, can be done by:

nested_list = [line[:-1].split(';') for line in lines] 

You can then iterate over the inner lists to check if they contain a 0:

filtered_list = [line for line in nested_list if '0' not in line]

Then we need to convert everything to floats:

result = [list(map(float, line)) for line in filtered_list]

Or if you really want to have a one-liner:

result = [list(map(float, line)) for line in [line[:-1].split(';') for line in lines] if '0' not in line]

Answer 2

[ x
  for x in ([ float(v) for v in line.split(';') if v ]
            for line in lines)
  if any(x[1:]) ]

We have an inner generator which iterates all lines, splits each line, removes the empty strings and converts all remaining values to floats. Then we have an outer loop which checks each of these results for containing only zeros by using the any() function on all but the first element.

This produces all floats. If you need only the first element in each line to be a float and the rest shall be integers, then use this extension:

[ x
  for x in ([ (int if i else float)(v)
              for i, v in enumerate(line.split(';'))
              if v ]
            for line in lines)
  if any(x[1:]) ]