selenium works on local and not on azure server

Question

I am trying to get video url from links on this page. Video link could be seen on https://in.news.yahoo.com/video/jaguar-fighter-aircraft-crashes-near-084300217.html . (Open in Chrome)

For that I wrote chrome web driver related code as below :

from bs4 import BeautifulSoup
from selenium import webdriver
from pyvirtualdisplay import Display

chromedriver = '/usr/local/bin/chromedriver'
os.environ['webdriver.chrome.driver'] = chromedriver
display = Display(visible=0, size=(800,600))
display.start()
driver = webdriver.Chrome(chromedriver)

        driver.get('https://in.news.yahoo.com/video/jaguar-fighter-aircraft-crashes-near-084300217.html')
        try:
            element = WebDriverWait(driver, 20).until(lambda driver: driver.find_elements_by_class_name('yvp-main'))
            self.yahoo_video_trend = []
            for s in driver.find_elements_by_class_name('yvp-main'):
                print "Processing link  - ", item['link']
                trend = item
                print item['description']
                trend['video_link'] = s.find_element_by_tag_name('video').get_attribute('src')
                print 
                print s.find_element_by_tag_name('video').get_attribute('src')
                self.yahoo_video_trend.append(trend)
        except:
            return

This works fine on my local system but when I run on my azure server it does not give any result at s.find_element_by_tag_name('video').get_attribute('src')

I have installed chrome on my azureserver.

Update :

Please see, requests and Beautifulsoup I already tried, but as yahoo loads html content dynamically from json, I could not get it using them.

And yeah azure server is simple linux system with command line access. Not any application.

Answer 1

I tried to reproduce your issue using you code. However, I found there was no tag named video in that page(' https://in.news.yahoo.com/video/jaguar-fighter-aircraft-crashes-near-084300217.html ')(using IE and Chrome to test). I used the developer Tool to check the HTML code, like this picture:

It seems that this page used the flash player to play video,not HTML5 video control. For this reason, I suggest that you can check your code whether used the rightly tag name. Any concerns, please feel free to let me know.

Answer 2

We tried to reproduce the error on our side. I was not able to get chrome driver to work, but I did try the firefox driver and it worked fine. It was able to load the page and get the link via the URL.

Can you change your code to print the exception and send it to us, to see where the script is failing?

Change your code:

except:
    return

try

do

except Exception,e: print str(e)

Send us the exception, so we can take a look.