boto3是否可以使用readlines？

Question

I'm trying to run a diff on two files that are stored in S3, and would like to avoid downloading the files if possible.

The sample code I am working with is as so:

import difflib

file1 = open('sample1.csv', 'r');
file2 = open('sample2.csv', 'r');

diff = difflib.ndiff(file1.readlines(), file2.readlines())

I see with boto3 package, I can open the file from S3, but how can I pass the equivalent of file1.readlines() and file2.readlines() into the ndiff function?

Answer 1

For future readers, I'll answer the exact question "Is it possible to use readlines with boto3?"

import io

// import stuff and set up s3_client

body = s3_client.get_object(Bucket=bucket, Key=key)['Body']
stream = io.BufferedReader(body._raw_stream)
stream.readlines()

As indicated by comments on the question, readlines() pulls everything into memory, which is why you can pass a hint to it so "no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds hint." ( https://docs.python.org/2/library/io.html#io.IOBase.readlines )