ValueError：无法将字符串转换为float ---如何将字符串列表转换为numpy数组类型float？

Question

It looks like I have a malformed numpy array in Python3.x---this was saved as a list of lists of strings.

foo = [[7.0352220e-01 5.3130367e-06 1.5167372e-05 1.0797821e-06]
  [1.3130367e-06 2.4584832e-01 2.2375602e-05 7.3299240e-06] [7.2646574e-06 7.1252006e-06 3.0184277e-01 ... 1.0048618e-05 3.1828706e-06 1.0196264e-06]..]

I get the following error trying to read in this data as np.float32 into a numpy array:

np.asarray(foo, dtype=np.float32)

error:

ValueError: could not convert string to float:[[7.0352220e-01 5.3130367e-06 1.5167372e-05 1.0797821e-06][1.3130367e-06 2.4584832e-01 2.2375602e-05 7.3299240e-06] [7.2646574e-06 7.1252006e-06 3.0184277e-01 ... 1.0048618e-05 3.1828706e-06 1.0196264e-06]..]

I've tried explicitly converting each list element into a float as follows:

try2 = np.asarray(map(np.float32, foo))

but it snags on a bracket:

ValueError: could not convert string to float: [

What is the recommended way to convert a list of lists of strings into a numpy array, type float?

Answer 1

If you replace the spaces with commas, you can use json.loads to read the string as a list, and pass that to np.asarray :

import json
import numpy as np

foo = "[[7.0352220e-01 5.3130367e-06 1.5167372e-05 1.0797821e-06] \
[1.3130367e-06 2.4584832e-01 2.2375602e-05 7.3299240e-06]]"

a = np.asarray(json.loads(foo.replace(" ", ",")), dtype=np.float32)
print(a)
#array([[7.0352220e-01, 5.3130367e-06, 1.5167372e-05, 1.0797821e-06],
#       [1.3130367e-06, 2.4584832e-01, 2.2375602e-05, 7.3299240e-06]])

print(a.dtype)
#float32

This assumes there is exactly 1 space between values. If that is not the case, you can use re.sub to replace multiple spaces with a comma:

import re
a = np.asarray(json.loads(re.sub("\s+", ",", foo)))
#array([[7.0352221e-01, 5.3130366e-06, 1.5167372e-05, 1.0797821e-06],
#       [1.3130367e-06, 2.4584831e-01, 2.2375601e-05, 7.3299238e-06]],
#      dtype=float32)

Answer 2

As far as I have seen, np.asarray() works only if dtype has a different datatype from the initial datatype. Please try and remove that argument and see if it works.

Answer 3

How is your string data shaped? Probably the simplest way is to use split() and iterate over the list. Example (list of lists of strings) that worked for me:

foo = [['7.0352220e-01 5.3130367e-06 1.5167372e-05 1.0797821e-06'],
       ['7.0352220e-01 5.3130367e-06 1.5167372e-05 1.0797821e-06']]
arr = np.array([[value.split() for value in row][0] for row in foo], dtype='<f8')

(Note: the [0] is used as split creates a list itself. You can use np.reshape in alternative)

EDIT: if its a string representation (not a list of strings as stated in the OP):

foo = '[[7.0352220e-01 5.3130367e-06 1.5167372e-05 1.0797821e-06][7.0352220e-01 5.3130367e-06 1.5167372e-05 1.0797821e-06]'
arr=np.array([line.split() for line in foo.replace('[','').replace(']]','').split(']')], dtype='<f8')

Answer 4

Given:

foo = [['7.0352220e-01 5.3130367e-06 1.5167372e-05 1.0797821e-06'],
       ['1.3130367e-06 2.4584832e-01 2.2375602e-05 7.3299240e-06'], 
       ['7.2646574e-06 7.1252006e-06 3.0184277e-01 1.0048618e-05']]

Try this to split each string

foo = [row[i].split() for row in foo for i in range(len(foo[0]))]

This for changing type to floats.

foo = [[float(row[i]) for i in range(len(foo[0]))] for row in foo]

print(type(foo[0][1]))

>> float

Then turn it into a numpy array:

foo = np.array(foo)

print(type(foo[0][1]))

>> numpy.float64