读取url内容，在python中进行操作

Question

I'm trying to get data from URL containing space-separated value line by line and perform some operations on each line.

The URL contains:

586763334343 1523718907817 NullPointerException
586763434443 1523718907818 IllegalAgrumentsException
....

Here, 1st string is the request-id , 2nd string is a timestamp (in UTC) and the 3rd string is the Exception Name .

I'm trying to read each line from URL and convert the timestamp to date-time and append to a list such that my list looks something like this.

[
 { second: 12, minute: 27, hour: 21, data: 'NullPointerException' },
 { second: 12, minute: 27, hour: 21, data: 'NullPointerException' }
]

How can I read the url and perform convert timestamp into date-time and append it to a list?

I tried:

url = request.GET.get('url')
with closing(requests.get(url, stream=True)) as f:
    data = [list(map(int, line.split())) for line in f]
print(data)

I'm trying to read url in the following manner and I can't come up with solution to perform other operations.

Answer 1

Use,

from datetime import datetime
import requests

url = "https://example/log1.txt"

resp = requests.get(url)
if resp.status_code == 200:
    data = []
    for line in resp.text.splitlines():
        split_ = line.split()
        # divide timestamp by 1000 to convert from milliseconds to seconds.
        date = datetime.utcfromtimestamp(int(split_[1]) / 1000)
        data.append({
            "second": date.second,
            "minute": date.minute,
            "hour": date.hour,
            "data": split_[-1],
        })

    print(data)