Why doesn’t input.send_keys() work in my Selenium WebDriver Python script when run as www-data?

Question

I have a Python script that uses Selenium WebDriver (with PyVirtualDisplay as the display) to log into Flickr.

• http://pastebin.com/dqmf4Ecw (you'll need to add your own Flickr credentials)

When I run it as myself on my Debian server, it works fine. (I'm a sudoer, but I don't use sudo when running the script.)

When I run it as the user www-data (which is what it'll be running as eventually, because I want to trigger it from a Django website), I get two problems, one small, one big:

1. (Small): the webdriver.Firefox() call takes 30–45 seconds to return, compared to 2 seconds when run as myself
2. (Big): the script fails to log into Flickr. In order to log in, I find the username and password fields on the Flickr signin page (http://www.flickr.com/signin/), and use element.send_keys() to enter the username and password. Although Selenium seems to find the elements (ie no NoSuchElementException is thrown), the values do not get entered in the fields when the script is run as www-data (according to the screenshots I take using browser.save_screenshot ), unlike when the script is run as myself.

Why does send_keys() not work when the script is run as www-data ? (And is it related to the browser taking much longer to start?)

Answer 1

Maybe you have something different in your environment.

Try copy by example your ~/.bashrc in /home/www-data

If it's not sufficient, run this command both as your current user & as www-data :

strace -tt -f -s 1000 -o /tmp/trace ./script.py

And paste it (filter out your logins/passwords) somewhere.

We will see what's happens.

Answer 2

Sometimes, Firefox performs some nasty plugin compatibility check during startup. As each user can have a different set of browser plugins, this could be responsible for the difference in startup times. You could try to sync your Firefox profiles between users.

Then, are you sure that Firefox as user www-data has proper network/internet access? Can you confirm that the Flickr site loads properly via SeleniumHQ? " The script fails to log into Flickr " is too unprecise. Some more details about why it fails might reveal the problem instantaneously.

Edit: Sorry, I just understood that there shouldn't be a difference in profiles, because Selenium creates one. Nevertheless, my second point might be useful, so I won't delete this answer.

Answer 3

Some more things to ponder about:

• Could you spawn firefox manually from www-data account once and make sure that Firefox is not updating itself before every execution of the script? I once faced this problem with Selenium RC on Windows and had to let the update finish before starting the script with the updated binary.

• As a workaround, I guess you could you try running the script as www-data user but connecting remotely to a webdriver server running in your login (aka "grid" mode). Would that work for you?

Answer 4

I would suggest getting the latest chrome from google and trying input.send_keys() in that browser instead.

Sometimes some features of webdriver get broken with new releases.. If you are bent on testing with firefox, you might have better luck with an older/newer version of selenium webdriver.

I remember having a similar issue regarding send_keys() on a mac.. My issue was that send_keys() did not work in certain modal windows after I updated selenium webdriver.. I fixed it by reverting to an older webdriver that I knew to work. However, I was using Ruby and not Python to drive webdriver.

sometimes, there might also be a problem with getting the correct ENV variables in your shell if you use it as a different user. I would suggest trying to troubleshoot and see if all the shell ENV variables are set properly under www-data.