Get JavaScript variable using Mechanize

Question

I want to get a JavaScript variable from https://admin.booking.com/hotel/hoteladmin in head > script > var token .

I don't know how this variable is set by the browser because when I get this page from Mechanize I get:

var token = '' || 'empty-token',

Here is the code I use to GET this page:

login_url = "https://admin.booking.com/hotel/hoteladmin"
agent = Mechanize.new
agent.verify_mode= OpenSSL::SSL::VERIFY_NONE
page = agent.get(login_url)

Answer 1

If you want to access this token via JavaScript in mechanize/watir, you need to be able to access it with your browsers developers tools as well.

Unfortunately the variable itself is enclosed in a scope, which makes it impossible to access it just like that. You can read more about the many different types of scopes in JS in this excellent post: What is the scope of variables in JavaScript?

Now to answer your question. Sure, it would be possible to extract the token itself, but you would have to do it in a dirty manner. You would have to wait untill all JS is executed on the page and then take the document body and extract it somehow - one way could be RegExp.

EDIT: Mechanize does according to a a few other answers here on SO not execute JS, which is why you need a gem that drives a browser (watir is a great example), but that drives us back to the problem described above.