将会话ID从Selenium传递到Python请求

Question

100% Python noob, so I apologize if any terms or phrases I use are incorrect or ambiguous

I am trying to go to a random word generator website, refresh the random words generated from the default, scrape the website after the refresh, ingest the words, sort them alphabetically, then print the pre-sorted and post-sorted lists to the screen.

This is the website url: https://www.randomlists.com/random-words

I am using the latest version of Python 3.x, which comes with Requests, and I have installed Selenium using Pip.

Here's what I have been able to successfully do:

1. Use Selenium to pass a refresh command to the website and generate a new list of words
2. Use Requests to scrape the default words from the website.
3. Sort the default word list alphabetically
4. Print the pre-sorted and post-sorted default word lists to the screen

Here's what I can't do that I want to do:

1. Use Selenium to refresh the website from the default
2. Then use Requests to scrape the refreshed words from the website
3. Then sort alphabetically and print to the screen, pre-sort and post-sort.

I've discovered that the reason for this is that Selenium and Requests are each using their own "instances", represented by Session IDs, of the website.

So (finally) here is my question - how do I pass the captured Session ID from the website refreshed with Selenium to Requests which I then use to scrape the refreshed word list rather than the default word list?

All of the topics around this issue have dealt with passing a username and password login session from one to the other, which isn't what I'm trying to do here. My apologies if this question has been already asked - I spent several hours today researching, but was not able to find anything with my, specific scenario.

More than happy to post a copy of my current code if necessary.

Thanks!

Edit to add code below

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
options.binary_location = "/usr/bin/chromium"
driver = webdriver.Chrome(executable_path=r"REDACTED")
url = 'https://www.randomlists.com/random-words'
driver.get(url)

refresh_button = driver.find_elements_by_xpath("//input[@name='submit' and 
@value='Refresh']")[0]
refresh_button.click()

rawList = []
sortedList= []
url = 'https://www.randomlists.com/random-words'
r = requests.get(url)
tree = lxml.html.fromstring(r.content)
elements = tree.get_element_by_id('result')
for el in elements:
    rawList.append(el.text_content())

print("This is the unsorted list:", "\n")
for i in rawList:
    print(i)

print("\n")
print("This is the sorted list:", "\n")
for i in rawList:
    sortedList.append(i)
sortedList.sort()
for i in sortedList:
    print(i)

Answer 1

I am trying to go to a random word generator website, refresh the random words generated from the default, scrape the website after the refresh, ingest the words, sort them alphabetically, then print the pre-sorted and post-sorted lists to the screen.

I'm not sure if this is an assignment or anything and has to be done in this specific way but there may be an easier way. Looking at the requests on that website, I found the link to the JSON of all the words they seem to be choosing random ones from. Now, they don't seem to be sorted but that is as simple as calling sorted(...) on the returned data.

>>> import requests
... 
... url = 'https://www.randomlists.com/data/words.json'
... r = requests.get(url)
... r.raise_for_status()
... all_words = r.json()['data']
>>> len(all_words)
2643
>>> all_words == sorted(all_words)
False
>>> sorted_words = sorted(all_words)