AWS Lambda + API Gateway response non-utf8 CSV

Question

Requirement:

• Use AWS Lambda + API Gateway
• Function language: Python
• Convert a text from UTF-8 to Shift_JIS (such as: 武蔵野冷蔵名古屋)
• Response with csv format (application/csv)

Issue:

• I'm using response = codecs.encode(inputString, 'SHIFT_JIS', errors='ignore') with inputString something like武蔵野冷蔵名古屋: encode input string with shift_jis
• And I return response to user. But facing error Invalid lambda response received: Invalid API Gateway Response Keys: {'requestId', 'errorMessage', 'stackTrace', 'errorType'} in {'errorMessage': "Unable to marshal response: 'utf-8' codec can't decode byte 0x95 in position 0: invalid start byte" .

Seem that API Gateway just support UTF-8 for text.

My concern is how to response non UTF-8 text when combine Lambda + API Gateway.

• I'm checking the guide setup Binary data with API Gateway but not work now
• I don't want to base64.encode it and decode later at client side.

Any help for this issue? Thanks!

Answer 1

Thank God!

I was guided to solve this issue.

Just base64 encode the string_after_encode_to_shift_jis.

return {
            'statusCode': 200,
            'headers': {
                'Access-Control-Allow-Origin': '*',
                'Content-Type': f'application/csv; charset=shift_jis'
            },
            'body': base64.b64encode(data.encode(enc)).decode('utf-8'),
            'isBase64Encoded': True
        }