从 lambda 中的 s3 读取 doc、docx 文件

Question

TLDR; reading with my AWS lambda doc , docx files that are stored on S3.

On my local machine I just use textract.process(file_path) to read both doc and docx files.

So the intuitive way to do the same on lambda is to download the file from s3 to the local storage ( tmp ) on the lambda and then process the tmp files like I do on my local machine.

That's not cost-effective...

Is there a way to make a pipeline from the S3 object straight into some parser like textract that'll just convert the doc / docx files into a readable object like string ?

My code so far for reading files like txt.

import boto3

print('Loading function')


def lambda_handler(event, context):
    try:  # Read s3 file
        bucket_name = "appsresults"
        download_path = 'Folder1/file1.txt'
        filename = download_path
        s3 = boto3.resource('s3')
        content_object = s3.Object(bucket_name, filename)        

        file_content = content_object.get()['Body'].read().decode('utf-8')

        print(file_content)

    except Exception as e:
        print("Couldnt read the file from s3 because:\n {0}".format(e))

    return event  # return event

Answer 1

This answer solves half of the problem

textract.process currently doesn't support reading file-like objects . If it did, you could have directly loaded the file from S3 into memory and pass it to the process function.

Older version of textract internally used python-docx package for reading .docx files. python-docx supports reading file-like objects. You can use the below code to achieve your goal, at least for .docx files.

import boto3
import io
from docx import Document

s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket')
object = bucket.Object('/files/resume.docx')

file_stream = io.BytesIO()
object.download_fileobj(file_stream)

document = docx.Document(file_stream)

Answer 2

If you're reading the docx file from s3, Document() constructor expects path for the file. Instead, you can read the file in byte format and call the constructor like this.

from docx import Document

def parseDocx(data):
    data = io.BytesIO(data)
    document = Document(docx = data)
    content = ''
    for para in document.paragraphs:
        data = para.text
        content+= data
    return content

Key = "acb.docx"
Bucket = "xyz"
obj_ = s3_client.get_object(Bucket= Bucket, Key=Key)
if Key.endswith('.docx'):
    fs = obj_['Body'].read()
    sentence = str(parseDocx(fs))