How to extract whole data of a website using python?

Question

I am trying to read data of tables from the following website: https://cdn.ime.co.ir , using the following code:

import requests
import urllib.request
import time
from bs4 import BeautifulSoup

url = 'https://cdn.ime.co.ir'
response = requests.get(url)

soup = BeautifulSoup(response.text, "html.parser")

But when I check the content of soup the data of the tables that I need to read aren't in there!

This is one of the tables that I like to read it's data that will be shown by clicking on it's above bar.

Answer 1

I would recommend you for getting the website data, start with a GET request instead of POST .

response = request.GET(<url>)

Differences between GET and POST

After that, you should use beautifulsoup4 for analysing the data inside the response:

BS4 Documentation

Regards!