How to pass special parameter into os.system in Python3?

Question

I am passing parameter into os.system in Python3 like this:

os.system("scrapy crawl %s -a arg='%s'" % ("googlebook",scrapy_url))

the scrapy url contains special charactor:

'?q=19434&maxResults=40&startIndex=200'

In the Spider only recieve:

?q=19434

How to pass special words into spider?

Answer 1

The shlex module knows how to escape strings you want to pass to the shell.

However, the shell isn't doing anyting useful here, so it's better to simply avoid it.

from subprocess import run
run(["scrapy", "crawl", "googlebook", "-a", "arg=" + scrapy_url],
    # This will raise an exception if scrapy fails
    # which is _probably_ a good idea in most scenarios
    check=True)

Of course, given that scrapy is a Python library, perhaps the best solution is to simply import scrapy and take it from there.