Getting error while Webscraping for a requests.post method

Question

I am trying to extract the data for a state office in "DELHI'. However, my code is not working. I am sure the data parameters are incorrect in my python code. I have imported all the required libraries like pandas, beautifulSoup, requests etc before running the code.

r = requests.get('https://search.epfindia.gov.in/locate_office/office_location.php')
def get_all_forms(url):
    soup = BeautifulSoup(r.content, "html.parser")
    return soup.find_all('form')
details = {}
action = form.attrs.get("action").lower()
method = form.attrs.get("method", "get").lower()
State_value = "Delhi"
district_value = "East Delhi"
Pin_value = "110032"
inputs = [State_value, district_value, Pin_value]
fetchdata = requests.post(form, data = inputs)
print (fetchdata.text)

The website looks like this: https://search.epfindia.gov.in/locate_office/office_location.php

In the form, there are a State/UT, District and PIN/Area field. Each State has respective districts. PIN/Area field is not visible, but for some States like "DELHI", after selecting District field, the PIN/Area field appears and we need to select appropriate PIN Code. After selecting the options from dropdown, we need to submit the form and it gives a filtered table for selected options.

I am trying to extract all the office address present in a districts of a State. Please help me building the code. If you write me the code, then I'll study the code and understand where I went wrong. Otherwise if there is any study material on such type of webscraping through web-form post method please tell me. I will study them and try again. Thank you.

Answer 1

To get data for specific PIN you can use this example:

import requests
from bs4 import BeautifulSoup


post_url = "https://search.epfindia.gov.in/locate_office/resulttable.php"
data = {
    "submit1": "submit1",
    "state": "DELHI",
    "district": "EAST DELHI",
    "pin_area": "110032",
}

soup = BeautifulSoup(requests.post(post_url, data=data).content, "html.parser")

for td in soup.select("td.large_font"):
    print(td.text)

Prints:

DSIIDC Facility Centre Building, Flatted Factory Complex,
2nd & 3rd Floor, Jhilmil Industrial Area,
New Delhi, DELHI
Email: sro.laxminagar@epfindia.gov.in

Bhavishya Nidhi Bhawan, 8 th Floor, 28,Community Centre,
Wazirpur Industrial Area,
Delhi, DELHI
Email: acc.dlut@epfindia.gov.in