python read file with block, but end with newline(\n)

Question

test.txt is a "\\n" split text file:

f = open('test.txt','r') f.read(256)

But while read 256, the last records may not with full line.

How to read such as:

I set read 256 but when 248 is the "\\n" and 256 the last records not with full line just read 248, and f.tell() give the 248 position.

Thanks.

Answer 1

If you're using newlines to split your data, why not read it in the same way?

with open('test.txt', 'r') as f:
    lines = f.readlines()
# Now each line in lines is a complete line.

Answer 2

What you want to do is read complete lines. For this task, you usually do something of this effect.

size_so_far = 0
contents = []

for line in open('test.txt'):
      size_so_far += len(line)
      if size_so_far > 256:
         break
      contents.append(line)

contents = "".join(contents)

Answer 3

The simplest way to read a file with variable-length lines separated by any of '\\n' , '\\r' or '\\r\\n' or even a mixture of those is:

with open('yourfile.txt', 'rU') as f:
    for line in f:
        do_something_with(f)
        # optional, if you want to bale out after 256 bytes:
        if f.tell() >= 256: break

What that does is read large blocks, find the line endings, and yield a line at a time. The underlying code is written in C. I have yet to see any evidence that doing the same thing in Python code would be faster.

Answer 4

do you care about efficiency?

here is one way to do it:

data=f.read(256)
data=data.splitlines(True)
if data[-1]!=data[-1].splitlines()[-1]:
    #must be newline at end of last line
    data="".join(data)
else:
    data="".join(data[:-1])

print data