简体   繁体   中英

Ruby + watir: format cookie for web crawler

I have started building a web scraper to get some data for an API, however the url requires a 'continue as guest' for each daily visit. This is the cookie I from the request header:

web_site=eyJpdiI6Im0ySWpxMFdLVW56RVk0NzU5cEZNZlE9PSIsInZhbHVlIjoiWU5XVWNzZTlMejEzOHFhQ0hsSkVlbmZxbm5zeWNuU0tGZ1hNOUNuUGZQdFFJUDB2M2M4NFRzbGx2WjdMZ1NTQ0dUZ0ZqRmNUVmk5XC9JeThVTlhVZm1RPT0iLCJtYWMiOiI4ZTBhZjUxZWFmNDhiZjI0MzE1OGMyNjY2N2EyOTkyODM4N2NlYWIzZTBmNmI2ZmExMDg5ZTcyYTY3MzM2MWZiIn0%3D; expires=Tue, 01-Mar-2022 20:00:34 GMT; Max-Age=7200; path=/;samesite=none;Secure; domain=.website.co.uk

So i am trying to add this cookie to my request but can't seem to get the formatting right (or am doing something else wrong).

This

browser.cookies.add('openplay_session', 'eyJpdiI6Ill6RFRPeXJzcktzcndCM3pEM1I0S3c9PSIsInZhbHVlIjoiN1pCRDZjaXRBbUpSaU1BOFhEVkVld0tsWXlcL0l6RlNISmdoQ25JWnpcL3BxNVREWWFaWk9kY1wvTDVFMUI0aW81TmptN0ppTFlISkxMclhianB1WlFRWnc9PSIsIm1hYyI6IjE1NTVjZjJjZTQzYzQ0ODdmZTdlODY2ZDBjODdjNzYyY2Q5ZGJmMDYxNjIyOGUzZjU3N2JjZWYwMjRjZjVjNzUifQ%3D%3D', path='/', Secure = true, domain='.openplay.co.uk')

returns:

`add': wrong number of arguments (given 5, expected 2..3) (ArgumentError)

Full code so far (not complete as I cant access the tables content w/o the cookie):

require 'watir'
require 'webdrivers'
require 'nokogiri'
require 'securerandom'   

browser = Watir::Browser.new
browser.goto 'https://www.website.co.uk/booking'
browser.cookies.add('open_session', 'eyJpdiI6Ill6RFRPeXJzcktzcndCM3pEM1I0S3c9PSIsInZhbHVlIjoiN1pCRDZjaXRBbUpSaU1BOFhEVkVld0tsWXlcL0l6RlNISmdoQ25JWnpcL3BxNVREWWFaWk9kY1wvTDVFMUI0aW81TmptN0ppTFlISkxMclhianB1WlFRWnc9PSIsIm1hYyI6IjE1NTVjZjJjZTQzYzQ0ODdmZTdlODY2ZDBjODdjNzYyY2Q5ZGJmMDYxNjIyOGUzZjU3N2JjZWYwMjRjZjVjNzUifQ%3D%3D', path='/', domain='.website.co.uk')

parsed_page = Nokogiri::HTML(browser.html)

File.open("parsed.txt", "w") { |f| f.write "#{parsed_page}" }

    puts parsed_page.title

    links = parsed_page.css('a')
    links.map {|e| e["href"]}
    puts links

browser.close

Those last 3 parameters need to be inside a Hash.

browser.cookies.add('openplay_session', value, {path: '/', secure: true, domain: '.openplay.co.uk'})

https://github.com/watir/watir/blob/main/lib/watir/cookies.rb#L40-L55

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM