[英]ConnectionResetError: [Errno 104] Connection reset by peer and ERR_NAME_NOT_RESOLVED on heroku with mobile testing through Selenium
I'd like to test multiple mobile user agents with selenium and chrome. 我想用硒和铬测试多个移动用户代理。 I'm using python 3.6 and deploying to heroku.
我正在使用python 3.6并部署到heroku。 Based on http://chromedriver.chromium.org/mobile-emulation .
基于http://chromedriver.chromium.org/mobile-emulation 。
You can download my project for both windows and heroku use at: 您可以在以下位置下载适用于Windows和Heroku的项目:
https://github.com/kc1/mobiletest
(keep in mind that if you deploy to heroku you have to set FLASK_CONFIG to production. Also please note that the code in the project is slightly different than in this question because I've been playing with the code over the past week.) (请记住,如果您部署到heroku,则必须将FLASK_CONFIG设置为生产环境。也请注意,该项目中的代码与该问题略有不同,因为我在过去一周中一直在使用这些代码。)
I have: 我有:
def some_long_calculation():
driver = create_chromedriver('kkk')
# driver = create_chromedriver()
driver.get("https://www.yahoo.com/")
.....
and : 和:
def create_chromedriver(ua=False):
options = webdriver.ChromeOptions()
CHROMEDRIVER_PATH = os.getenv('$HOME') or basedir+'/chromedriver.exe'
FLASK_CONFIG = os.getenv('FLASK_CONFIG')
if ua:
mobile_emulation = {"deviceName": "Nexus 5"}
options.add_experimental_option("mobileEmulation", mobile_emulation)
if FLASK_CONFIG and FLASK_CONFIG == "production":
CHROMEDRIVER_PATH = '/app/.chromedriver/bin/chromedriver'
GOOGLE_CHROME_SHIM = os.getenv('$GOOGLE_CHROME_SHIM') or 'no path found'
options.binary_location = '/app/.apt/usr/bin/google-chrome-stable'
options.add_argument('--disable-gpu')
options.add_argument('--no-sandbox')
return webdriver.Chrome(executable_path=CHROMEDRIVER_PATH, options=options)
If I run it locally with the mobile browser enabled It works as expected: 如果在启用了移动浏览器的情况下在本地运行它,则按预期运行:
If I run it on heroku with the mobile browser enabled : 如果我在启用了移动浏览器的heroku上运行它:
Then I tried it on heroku with the mobile user disabled I get: 然后我在禁用移动用户的情况下在heroku上尝试了以下操作:
So at least I know the setup is working as far as chrome and chromedriver. 所以至少我知道该设置在chrome和chromedriver上都可以正常工作。
heroku Logs: heroku日志:
2018-07-15T17:37:53.967643+00:00 app[web.1]: driver = create_chromedriver('kkk')
2018-07-15T17:37:53.967637+00:00 app[web.1]: png = some_long_calculation()
2018-07-15T17:37:53.967645+00:00 app[web.1]: File "/app/app/main/cl.py", line 120, in create_chromedriver
2018-07-15T17:37:53.967640+00:00 app[web.1]: File "/app/app/main/cl.py", line 123, in some_long_calculation
2018-07-15T17:37:53.967648+00:00 app[web.1]: return webdriver.Chrome(executable_path=CHROMEDRIVER_PATH, options=options)
2018-07-15T17:37:53.967651+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 75, in __init__
2018-07-15T17:37:53.967654+00:00 app[web.1]: desired_capabilities=desired_capabilities)
2018-07-15T17:37:53.967656+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 156, in __init__
2018-07-15T17:37:53.967659+00:00 app[web.1]: self.start_session(capabilities, browser_profile)
2018-07-15T17:37:53.967661+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 251, in start_session
2018-07-15T17:37:53.967669+00:00 app[web.1]: response = self.command_executor.execute(driver_command, params)
2018-07-15T17:37:53.967664+00:00 app[web.1]: response = self.execute(Command.NEW_SESSION, parameters)
2018-07-15T17:37:53.967667+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 318, in execute
2018-07-15T17:37:53.967672+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/selenium/webdriver/remote/remote_connection.py", line 472, in execute
2018-07-15T17:37:53.967674+00:00 app[web.1]: return self._request(command_info[0], url, body=data)
2018-07-15T17:37:53.967677+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/selenium/webdriver/remote/remote_connection.py", line 496, in _request
2018-07-15T17:37:53.967679+00:00 app[web.1]: resp = self._conn.getresponse()
2018-07-15T17:37:53.967682+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/http/client.py", line 1331, in getresponse
2018-07-15T17:37:53.967685+00:00 app[web.1]: response.begin()
2018-07-15T17:37:53.967687+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/http/client.py", line 297, in begin
2018-07-15T17:37:53.967695+00:00 app[web.1]: line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
2018-07-15T17:37:53.967690+00:00 app[web.1]: version, status, reason = self._read_status()
2018-07-15T17:37:53.967698+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/socket.py", line 586, in readinto
2018-07-15T17:37:53.967692+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/http/client.py", line 258, in _read_status
2018-07-15T17:37:53.967700+00:00 app[web.1]: return self._sock.recv_into(b)
2018-07-15T17:37:53.967712+00:00 app[web.1]: ConnectionResetError: [Errno 104] Connection reset by peer
How can I fix this? 我怎样才能解决这个问题?
EDIT: 编辑:
Thanks for your detailed answer. 感谢您的详细回答。 I've changed the code to incorporate the flags you mentioned.
我更改了代码以合并您提到的标志。 Chrome version is 67.0.3396.99.
Chrome版本是67.0.3396.99。 Chromedriver is 2.40 and selenium is 3.13.
Chromedriver是2.40,硒是3.13。 Unfortunately, there is no change in the result.
不幸的是,结果没有改变。 I'm still getting the same error.
我仍然遇到相同的错误。 As far as your stage 2 and 3 advice.
至于您的第2阶段和第3阶段建议。 I'm currently deploying to heroku so I do not have total control of the environment variables.
我当前正在部署到heroku,因此我无法完全控制环境变量。 Is there a way to make of these changes using python?
有没有办法使用python进行这些更改?
EDIT 2: 编辑2:
As I think about it more on https://sites.google.com/a/chromium.org/chromedriver/mobile-emulation the example uses 正如我在https://sites.google.com/a/chromium.org/chromedriver/mobile-emulation上所考虑的那样,该示例使用了
from selenium import webdriver
mobile_emulation = { "deviceName": "Nexus 5" }
chrome_options = webdriver.ChromeOptions()
chrome_options.add_experimental_option("mobileEmulation", mobile_emulation)
driver = webdriver.Remote(command_executor='http://127.0.0.1:4444/wd/hub',
desired_capabilities = chrome_options.to_capabilities())
Are you suggesting the browser is at ' http://127.0.0.1:4444/wd/hub ' 您是否建议浏览器位于“ http://127.0.0.1:4444/wd/hub ”
In general when a client terminates abruptly without closing the connection a RST packet
is sent by the TCP/IP stack of the underlying OS. 通常,当客户端突然终止而不关闭连接时,底层OS的TCP / IP堆栈会发送
RST packet
。 Python converts this into an exception with the text Connection reset by peer . Python将其转换为带有peer重置的Connection文本的异常。 As per your error stack trace it means that once
self._read_status()
was invoked (internally) Python assumed to receive something but the connection was suddenly dropped and Python informs you of this error by raising the exception: 根据您的错误堆栈跟踪,这意味着一旦(内部)调用了
self._read_status()
, Python假定会接收到某些东西,但连接突然中断, Python会通过引发异常来通知您该错误:
ConnectionResetError: [Errno 104] Connection reset by peer
The situation is somewhat similar to this expression : 这种情况有点类似于此表达式 :
"Connection reset by peer" is the TCP/IP equivalent of slamming the phone back on the hook.
“由对等方重置连接”是TCP / IP的等效功能,它可以将电话重新挂断。 It's more polite than merely not replying, leaving one hanging.
礼貌比不回信,挂个电话更礼貌。 But it's not the FIN-ACK expected of the truly polite TCP/IP converseur.
但这不是真正礼貌的TCP / IP对话者所期望的FIN-ACK。
There can be multiple probabilities behind this error as follows. 此错误背后可能有多个概率,如下所示。
A quick and a precise solution will be to add a few recommended ChromeOptions along with the existing one as follows: 一种快速而精确的解决方案是添加一些建议的ChromeOption和现有的ChromeOption ,如下所示:
options.add_argument("start-maximized"); // open Browser in maximized mode
options.add_argument("disable-infobars"); // disabling infobars
options.add_argument("--disable-extensions"); // disabling extensions
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage"); // overcome limited resource problems
and then 接着
return webdriver.Chrome(executable_path=CHROMEDRIVER_PATH, options=options)
Note : You need to remove the argument options.add_argument('--disable-gpu')
as it is applicable to windows OS only. 注意 :您需要删除参数
options.add_argument('--disable-gpu')
因为它仅适用于Windows OS。
A couple of points: 要点:
Python Chrome Mobile Emulation
the Key and Value pair seems to be "deviceName": "Google Nexus 5"
(not "deviceName": "Nexus 5"
) Python Chrome Mobile Emulation
的文档,“ 键和值”对似乎是"deviceName": "Google Nexus 5"
(不是"deviceName": "Nexus 5"
) You can tweak you code to invoke Remote()
through either of the following ways- 您可以通过以下两种方式来调整代码以调用
Remote()
Invoking Remote()
through DesiredCapabilities()
: 通过
DesiredCapabilities()
调用Remote()
DesiredCapabilities()
:
from selenium import webdriver # import DesiredCapabilities was missing in your program from selenium.webdriver.common.desired_capabilities import DesiredCapabilities mobile_emulation = { "deviceName": "Google Nexus 5" } chrome_options = webdriver.ChromeOptions() chrome_options.add_experimental_option("mobileEmulation", mobile_emulation) capabilities = DesiredCapabilities.CHROME capabilities = options.to_capabilities() driver = webdriver.Remote(command_executor='http://127.0.0.1:4444/wd/hub', desired_capabilities = capabilities)
You find a similar discussion on invoking Remote()
through ChromeOptions()
in How to add selenium chrome options to 'desiredCapabilities'? 您如何在如何向'desiredCapabilities'中添加硒化铬选项中找到了关于通过
ChromeOptions()
调用Remote()
的类似讨论?
Invoking Remote()
through ChromeOptions()
: 通过
ChromeOptions()
调用Remote()
ChromeOptions()
:
from selenium import webdriver mobile_emulation = { "deviceName": "Google Nexus 5" } chrome_options = webdriver.ChromeOptions() chrome_options.add_experimental_option("mobileEmulation", mobile_emulation) driver = webdriver.Remote(command_executor='http://127.0.0.1:4444/wd/hub', options=chrome_options)
You find a similar discussion on invoking Remote()
through ChromeOptions()
in Remote WebDriver UnreachableBrowserException: Could not start a new session 您可以在远程WebDriver中找到有关通过
ChromeOptions()
调用Remote()
的类似讨论UnreachableBrowserException:无法启动新会话
Webdriver.android
Webdriver.android
文档的链接 Getting started with Selendroid
Getting started with Selendroid
文档的链接。 If you are still seeing the error, perform the following upgradation/cleanup tasks: 如果仍然看到错误,请执行以下升级/清理任务:
driver.quit()
within tearDown(){}
method to close & destroy the WebDriver and Web Client instances gracefully. tearDown(){}
方法内调用driver.quit()
,以优雅地关闭和破坏WebDriver和Web Client实例。 @Test
. @Test
。 Looking out for a granular solution to specific error I stepped into Amazon S3 and "Connection Reset by Peer" where Garry Dolley summerizes the the cause of the problem to be a combination of the below mentioned factors: 在寻找针对特定错误的精细解决方案时,我进入了Amazon S3和“ Peer进行的连接重置” ,其中Garry Dolley将问题的原因总结为以下提到的因素的组合:
Linux kernels 2.6.17+ increased the maximum size of the TCP window/buffer, and this started to cause other gear to wig out, if it couldn't handle sufficiently large TCP windows.
Linux内核2.6.17+增加了TCP窗口/缓冲区的最大大小,如果它不能处理足够大的TCP窗口,则会开始引起其他麻烦。 The gear would reset the connection, and we see this as a “Connection reset by peer” message.
齿轮将重置连接,我们将其视为“对等连接重置”消息。
A pottential solution will be to put the following entries within /etc/sysctl.conf
: 一种可能的解决方案是将以下条目放在
/etc/sysctl.conf
:
Note : This fix is easy peasy, but it'll slow your maximum throughput inexchange of faster downloads. 注意 :此修补程序很简单,但会减慢最大下载速度,从而最大程度地降低了交换速度。
If applicable, ensure that /etc/hosts on your system contains the following entry : 如果适用,请确保系统上的/ etc / hosts包含以下条目:
127.0.0.1 localhost.localdomain localhost
Here are some of the related discussions: 以下是一些相关讨论:
Here are the references of this discussion: 以下是此讨论的参考:
The specific error message says 具体的错误信息说
android 's server IP address could not be found.
找不到android的服务器IP地址。
To me, that indicates that the browser is trying to lookup a DNS entry for android
, which isn't a valid TLD. 对我来说,这表明浏览器正在尝试为
android
查找DNS条目,这不是有效的TLD。 By chance, is the browser attempting to access http://android
, rather than something like https://www.google.com
? 偶然地,浏览器是否尝试访问
http://android
,而不是https://www.google.com
东西? I can replicate that same error message in my own Chrome by typing http://android
into the address bar. 通过在地址栏中输入
http://android
,可以在自己的Chrome中复制相同的错误消息。 This leads me to believe that specifying a proper url for the browser should resolve the issue. 这使我相信为浏览器指定正确的URL应该可以解决问题。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.