在Ubuntu中不使用Chrome驱动程序和Selenium的Chrome浏览器自动化

Question

I am currently trying to automate Chrome Browser (Not Chrome Driver) on Ubuntu for saving the thousands of pages without Chrome Driver and Selenium which are somehow prohibited by the site.

In Mac OS, AppScript can handle Chrome without Chrome Driver and Selenium. And I succeed in automation of the downloading the page. However, I didn't find the alternatives of AppScript in Ubuntu.

Thus, I use keyborad automation tool (xdotool) by referring automate-save-page-as . It enables me to open a single page and save it to the storage, but it's too slow, unstable and hard to understand the code.

Is there any plausible way to automate chrome browser without using Selenium and Chrome Driver in Ubuntu? Or could any one give some hints to open multiple pages at the same time using xdotool and save it to local after a few seconds?

Answer 1

I implement the solution for this problem. Check "ubuntu_automation_example_multiple.py".

https://github.com/jonghkim/browser-automation-beyond-firewall

I write two essential script files that are "save_page_as_multiple_open" and "save_page_as_multiple_save" by referring automate-save-page-as .

#-*- coding: utf-8 -*-
import os
import warnings
import time
warnings.filterwarnings('ignore')

def trick_open(url, fname):
    cmd = "./save_page_as_multiple_open '{}' --destination '{}'".format(url, fname)
    os.system(cmd)

def trick_save(url, fname):
    cmd = "./save_page_as_multiple_save '{}' --destination '{}'".format(url, fname)
    os.system(cmd)

if __name__ == "__main__":
    url = 'https://www.example.com'
    cwd = os.getcwd()

    for i in range(5):
        trick_open(url, cwd + "/example{}.html".format(i))

    time.sleep(5)

    for i in reversed(range(5)):
        print("Save Path: ", cwd + "/example{}.html".format(i))
        trick_save(url, cwd + "/example{}.html".format(i))

    cmd = "killall google-chrome"
    os.system(cmd)

In "save_page_as_multiple_open", it opens multiple url using xdotool. After then, "save_page_as_mutiple_save" saves each page and close the page in reverse order.