Python将撇号写入文件

Question

I'm converting a downloaded Facebook Messenger conversation from JSON to a text file using Python. I've converted the JSON to text and it's all looking fine. I need to strip the unnecessary information and reverse the order of the messages, then save the output to a file, which I've done. However, when I am formatting the messages with Python, when I look at the output file, sometimes instead of an apostrophe, there's â instead.

My Python isn't great as I normally work with Java, so there's probably a lot of things I could improve. If someone could suggest some better tags for this question, I'd also be very appreciative.

Example of apostrophe working: You're not making them are you?

Example of apostrophe not working: Itâs just a button I discovered

What is causing this to happen and why does not happen every time there is an apostrophe?

Here is the script:

#/usr/bin/python3

import datetime

def main():

    input_file = open('messages.txt', 'r')
    output_file = open('results.txt', 'w')

    content_list = []
    sender_name_list = []
    time_list = []

    line = input_file.readline()

    while line:
        line = input_file.readline()

        if "sender_name" in line:
            values = line.split("sender_name")
            sender_name_list.append(values[1][1:])

        if "timestamp_ms" in line:
            values = line.split("timestamp_ms")
            time_value = values[1]
            timestamp = int(time_value[1:])         
            time = datetime.datetime.fromtimestamp(timestamp / 1000.0)      
            time_truncated = time.replace(microsecond=0)
            time_list.append(time_truncated)    

        if "content" in line:
            values = line.split("content")
            content_list.append(values[1][1:])

    content_list.reverse()
    sender_name_list.reverse()
    time_list.reverse()

    for x in range(1, len(content_list)):
        output_file.write(sender_name_list[x])
        output_file.write(str(time_list[x]))
        output_file.write("\n")
        output_file.write(content_list[x])
        output_file.write("\n\n")


input_file.close()
output_file.close()

if __name__ == "__main__":
    main()

Edit: The answer to the question was adding

import codecs
input_file = codecs.open('messages.txt', 'r', 'utf-8')
output_file = codecs.open('results.txt','w', 'utf-8')

Answer 1

Without seeing the incoming data it's hard to be sure, but I suspect that instead of an apostrophe ( Unicode U+0027 ' APOSTROPHE ), you've got a curly-equivalent ( U+2019 ' RIGHT SINGLE QUOTATION MARK ) in there trying to be interpreted as old-fashioned ascii.

Instead of

output_file = open('results.txt', 'w')

try

import codecs
output_file = codecs.open('results.txt','w', 'utf-8')

You may also need the equivalent on your input file.