How to Scrape page with pagination with Python & Selenium

Question

I have been trying to scrap the table under the tab "Price History" from the website http://merolagani.com/CompanyDetail.aspx?symbol=ADBL

I have used Selenium to automate the process but cant actually find the actual result and cannot change to next page

Answer 1

EDIT: after few more test I see this url gives only data used by plot/graph on page, not data from "Price History" . I don't see url with "Price History" data so this answer doesn't resolve problem. It will need more digging in requests and code.

Page is created with ASP.Net which has very wierd system to send infromation to server.
Instead of links it uses JavaScript with <form> to send many information (in fields with name like _VIEWSTATE ).

---

JavaScript read data (as JSON) from urls like

http://merolagani.com/handlers/webrequesthandler.ashx?type=g‌​et_company_graph&sym‌​bol=ADBL&dateRange=1‌​2

so you can try do read it too

import requests

url = 'http://merolagani.com/handlers/webrequesthandler.ashx?type=get_company_graph&symbol=ADBL&dateRange=12'

r = r.requests(url)

data = r.json()

print('OK:', data['msgType'])
print('Symbol:', data['symbol'])
print('Name:', data['name'])
for row in data['quotes']:
    print('  date:', row['date'])
    print('  open:', row['open'])
    print(' close:', row['close'])
    print('  high:', row['high'])
    print('   low:', row['low'])
    print('volume:', row['volumen'])
    print('   rsi:', row['rsi'])
    print('----------------------')

Result:

OK: ok
Symbol: ADBL
Name: Agriculture Development Bank Limited
  date: 12/18/2016
  open: 540.0
 close: 540.0
  high: 540.0
   low: 525.0
volume: 6847.0
   rsi: 0.0
----------------------
  date: 12/19/2016
  open: 535.0
 close: 520.0
  high: 535.0
   low: 520.0
volume: 6963.0
   rsi: 0.0
----------------------
  date: 12/20/2016
  open: 520.0
 close: 520.0
  high: 530.0
   low: 505.0
volume: 9974.0
   rsi: 0.0
----------------------