简体   繁体   中英

How to send login payload to a form using relative URL for post method?

I'm trying to scrape a website which needs login. The problem is that the url of the login form is something like this:

/customer/guest/index

The question is how may I send the login information to this post method?

PS: I'm trying to use method mentioned in this article.

UPDATE: Here is the code I'm using

from bs4 import BeautifulSoup
import requests
import csv


# Start the session
session = requests.Session()

# Create the payload
payload = {'CustomerLogin[email]':'<USERNAME>',
          'CustomerLogin[password]':'<PASSWORD>',
          'csrf_token': '<TOKEN>'
         }

# Post the payload to the site to log in
s = session.post(" https://avangemail.net/customer/guest/index", data=payload)
print('Logging in...')

# Navigate to the next page and scrape the data
s = session.get('https://avangemail.net/customer/lists/all-subscribers')

print('Getting information...')
print(s.text)
soup = BeautifulSoup(s.text, 'html.parser')
table = soup.find('table')
tbody = table.find('tbody')

headers = [td.text for td in thead.select("tr td td")]

filename = 'avang-export'
with open(r'.'+filename+'.csv', "w") as f:
    wr = csv.writer(f)
    wr.writerows([[td.text for td in row.find_all("td")] for row in tbody.select("tr")])
    data= {
              '_username':'[YOUR_USERNAME]', 
              '_password':'[YOUR_PASSWORD]'
          }

then pass it to post method

response = requests.post(
    your_url,
    data=data
)

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM