如何使用自定义（多行）行终止符读取文本文件？

Question

I have a huge text file like this

1,2,3,4,5$*$*$2,5,1,3,2$*$*$

where $*$*$ is the line terminator (in reality this is needed because all kind of text may be included in the regular columns: newline characters, etc).

How can I parse the txt file efficiently and put it into a Pandas dataframe? pd.read_csv() only accept length-1 line terminators... so it fails here.

Here I am looking for

1,2,3,4
2,5,1,3

Thanks!

Answer 1

Maybe you can parse it before. Though I don't know any Pandas, I managed to make it work (I think):

with open("your_text_file.txt") as f:
    s = f.read()
parts = s.split("$*$*$")

df = pd.DataFrame(columns=['ONE', 'TWO', 'THREE', 'FOUR', 'FIVE'])
for i, line in enumerate(parts):
    if line:
        df.loc[i] = line.split(",")

print(df)

Answer 2

How about replacing your line terminator with one that pandas can understand?

from io import StringIO

s = '1,2,3,4,5$*$*$2,5,1,3,2$*$*$'
pd.read_csv(StringIO(s.replace('$*$*$', '\n')), header=None)

will return

   0  1  2  3  4
0  1  2  3  4  5
1  2  5  1  3  2