无法从网页中抓取一些附加到文本的时间戳

Question

I'm trying to scrape a timestamp attached to a text from a webpage. I can grab the text flawlessly but can't find out the timestamp. I can scrape the other timestamps attached to the comments from there, though. The timestamps which are with the comments can be found in the script tag as the value of created_at . However, I can't find the one I'm after.

website url

I've tried with:

import re
import json
import requests

url = 'https://www.instagram.com/p/CEuX_8iH95S/'

with requests.Session() as s:
    s.headers['User-Agent'] = 'Mozilla/5.0 (Windows NT 6.1; ) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36'
    r = s.get(url)
    script_tag = json.loads(re.findall(r"window\._sharedData = (.*?});",r.text)[0])
    post_content = script_tag['entry_data']['PostPage'][0]['graphql']['shortcode_media']['edge_media_to_caption']['edges'][0]['node']['text']
    print(post_content)

How can I parse the timestamp attached to the text from the site above?

Answer 1

You can parse the timestamp with .fromtimestamp() method from the datetime module.

Here's how to do it:

import datetime
import re
import json
import requests

url = 'https://www.instagram.com/p/CEuX_8iH95S/'

with requests.Session() as s:
    s.headers['User-Agent'] = 'Mozilla/5.0 (Windows NT 6.1; ) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36'
    r = s.get(url)
    script_tag = json.loads(re.findall(r'window\._sharedData = (.*?});', r.text)[0])
    post_date = script_tag['entry_data']['PostPage'][0]['graphql']['shortcode_media']['taken_at_timestamp']

    print(datetime.datetime.fromtimestamp(post_date).isoformat())
    print(datetime.datetime.fromtimestamp(post_date).strftime("%b %d %Y %H:%M:%S"))

This prints:

2020-09-04T20:25:49
Sep 04 2020 20:25:49

If you want to learn more about date formatting, check the docs here .