How to read from the network tab in Python?

Question

Using Selenium, Requests, & Beautiful Soup, I'd like to be able to locate & print this .m3u8 link (or any of the links shown here) but I don't know to show requests in Python.

def locator(url):
    driver = sp.driver # just geckodriver with profile
    driver.get(url)
    sleep(4)
    # from here needs to somehow access the network tab & locate GET requests with Host == "cfvod.kaltura.com"

Answer 1

Had to use ChromeDriver, PyChrome, and The DevTools Protocol, but this worked:

def outputstart(**kwargs):
    print("START ", kwargs)

driver = sp.driver # my chromedriver profile with an argument added for port 8000

dev_tools = pychrome.Browser(url="http://localhost:8000")
tab = dev_tools.list_tab()[0]
tab.start()

url = 'https://google.com'

start = time.time()
driver.get(url)
tab.call_method("Network.emulateNetworkConditions",
            offline=False,
            latency=100,
            downloadThroughput=93750,
            uploadThroughput=31250,
            connectionType="wifi")

def outputstart(**kwargs):
    print("START ", kwargs)

tab.call_method("Network.enable", _timeout=20)
tab.set_listener("Network.requestWillBeSent", outputstart)

Answer 2

https://stackoverflow.com/a/68363046/8491363

I used selenium (current release 3.0) to read the network log. Check out the link above.

One other way to do it is using Selenium 4.0 but it is still in beta state. When 4.0 gets released, it will officially support Chrome dev tool stuffs.