Timeout for webdriver.PhantomJS in python?

Question

I just use selenium-python.

When I use webdriver.Firefox() , I can get the result. When I use webdriver.PhantomJS() , the result can't be returned (the script is hanging).

Can anyone help me?

browser = webdriver.PhantomJS(executable_path='./lib/phantomjs/phantomjs')
url = "http://aminer.org/search/jie%20tang"
browser.get(url)

Answer 1

I've reproduced the issue and see hanging PhantomJS too. I've tried multiple workarounds (including loading the "https" url with disabled web security; try not to load images, increase the script and page load timeouts, update phantomjs etc), but haven't made it work till the moment.

There is an alternative approach here that doesn't involve selenium - use the AMiner API .

Here is how you can get the same search results using requests :

import requests


url = 'http://storeland.ru/user/login'
api_url = 'https://api.aminer.org/api/search/people'
with requests.Session() as session:
    session.get(url)

    params = {
        'query': 'jie+tang',
        'size': '20',
        'sort': 'relevance',
        # 'offset': 20  # set offset for pagination
    }
    response = session.get(api_url, params=params)
    for item in response.json()['result']:
        print item['name']

Prints (the first page of search results):

Zhu Jie-Tang
Jie-Tang Zhao
...
Jie-Tang Wu
Tian Jietang