How to read arrays to form a matrix from file using numpy

Question

I have a file with data as: 2 arrays in each row. Total= 10,000 rows.

[1,2,3,4,5][2,4,6,8,10]
[3,6,9,12,24][6,12,18,24,48]
....]

I am planning to give this input to Linear Regression in the fit command. I am having issue how to construct a matrix with entries.

I am looking at constructing Array (2 by x) like:

x=[
    [1,2,3,4,5]
    [3,6,9,12,24]
    ....]

y=
  [[2,4,6,8,10]
   [6,12,18,24,48]
    ....]

so that I can give to the fit command as input.

I see numpy.fromfile is used to get the binary data. can I use it for lists?

https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.fromfile.html

Answer 1

Here is Solution

f = open("test.txt","r")
x = []
y = []
for i in f.readlines():
    split_values = i.split("][")
    s1 = split_values[0].replace('[','')
    s2 = split_values[1].replace(']','')
    s1_split = s1.split(",")
    s2_split = s2.split(",")
    s1_split = map(int, s1_split)
    s2_split = map(int, s2_split)
    x.append(s1_split)
    y.append(s2_split)
print(x)
print(y)

Answer 2

Solution using pandas

import pandas as pd

df = pd.read_csv('input.txt', delimiter="\]\[", header=None, engine='python')
df[0] = (df[0] + ']')
df[1] = ('[' + df[1])
x = df[0].tolist()
y = df[1].tolist()