[英]Network unreachable when using Google Maps Geocode API in Google Dataflow
我在數據流作業中使用谷歌地圖地理編碼 API ( https://github.com/googlemaps/google-maps-services-java )。 我的 DoFn 在設置時准備 GeoApiContext。 過程元素 function 是這樣完成的:
public void processElement(ProcessContext c) {
String address = c.element().get("Address").toString();
String id = c.element().get("Id").toString();
Gson gson = new GsonBuilder().create();
try {
GeocodingResult[] results = GeocodingApi.newRequest(this.geocodeContext).address(address).language("pt-BR").components(ComponentFilter.country("BR")).await();
if(results.length == 0) {
TableRow outputRow = new TableRow();
outputRow.set("Id", id);
c.output(outputRow);
} else {
for(GeocodingResult r : results) {
TableRow outputRow = convertTableRow(gson.toJson(r).toString());
outputRow.set("Id", id);
c.output(outputRow);
}
}
} catch(ApiException e) {
LOGGER.error("ApiException on address: {}", address, e);
} catch(InterruptedException e) {
LOGGER.error("InterruptedException on address: {}", address, e);
} catch(IOException e) {
LOGGER.error("IOException on address: {}", address, e);
}
}
此代碼在本地運行良好,但在部署到數據流時會引發網絡錯誤:
exception: "java.net.ConnectException: Failed to connect to maps.googleapis.com/2607:f8b0:4001:c05:0:0:0:5f:443
at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:265)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:183)
at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:224)
at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.java:108)
at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.java:88)
at okhttp3.internal.connection.Transmitter.newExchange(Transmitter.java:169)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:41)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229)
at okhttp3.RealCall$AsyncCall.execute(RealCall.java:172)
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.ConnectException: Network is unreachable (connect failed)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
at java.base/java.net.Socket.connect(Socket.java:591)
at okhttp3.internal.platform.Platform.connectSocket(Platform.java:130)
at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:263)
... 22 more
我確保生成的虛擬機可以訪問互聯網,我什至可以從容器內部 ping maps.googleapis.com 端點:
USER@test-geocode-07020834-qmrj-harness-3k2l ~ $ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b2fd123138aa 3a1cb7aedd54 "/opt/google/dataflo…" 6 minutes ago Up 5 minutes k8s_healthchecker_dataflow-test-geocode-07020834-qmrj-harness-3k2l_default_5648e9815f2ca5beea8b0eb945e12d1f_0
086e36c3dd23 4127911f4769 "/opt/google/dataflo…" 6 minutes ago Up 5 minutes k8s_vmmonitor_dataflow-test-geocode-07020834-qmrj-harness-3k2l_default_5648e9815f2ca5beea8b0eb945e12d1f_0
2890fa415af5 664bd8972b23 "/opt/google/dataflo…" 6 minutes ago Up 6 minutes k8s_shuffle_dataflow-test-geocode-07020834-qmrj-harness-3k2l_default_5648e9815f2ca5beea8b0eb945e12d1f_0
eea757bf6be7 gcr.io/cloud-dataflow/v1beta3/beam-java11-batch "/opt/google/dataflo…" 6 minutes ago Up 6 minutes k8s_java-batch_dataflow-test-geocode-07020834-qmrj-harness-3k2l_default_5648e9815f2ca5beea8b0eb945e12d1f_0
b636784118f5 k8s.gcr.io/pause:3.1 "/pause" 6 minutes ago Up 6 minutes k8s_POD_dataflow-test-geocode-07020834-qmrj-harness-3k2l_default_5648e9815f2ca5beea8b0eb945e12d1f_0
lucas@test-geocode-07020834-qmrj-harness-3k2l ~ $ docker exec -it eea /bin/sh
# ping maps.googleapis.com
PING maps.googleapis.com (172.217.214.95) 56(84) bytes of data.
64 bytes from 172.217.214.95: icmp_seq=1 ttl=115 time=1.08 ms
64 bytes from 172.217.214.95: icmp_seq=2 ttl=115 time=1.28 ms
64 bytes from 172.217.214.95: icmp_seq=3 ttl=115 time=1.15 ms
64 bytes from 172.217.214.95: icmp_seq=4 ttl=115 time=1.41 ms
^C
--- maps.googleapis.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 1.089/1.235/1.414/0.131 ms
#
關於版本,我使用的是最新的梁版本(2.22.0)和最新的谷歌地圖版本(0.14.0)。
不知道還有什么可以看的,任何幫助表示贊賞。
更新
問題似乎是請求是使用 ipv6 地址完成的。 但是,GCE 機器似乎不支持 ipv6,並且調用只是失敗而沒有退回到 ipv4。
考慮到這一點,似乎沒有任何辦法解決這個問題:
謝謝
I had this exact same issue come up after upgrading from 2.17 to 2.24 and changing from Java 8 to Java 11. After trying to fix this on 2.24 and Java 11 I gave up and went back to 8 and it's working now.
我在任何地方都找不到它的文檔,但看起來使用的 userAgent 是基於此 -
當我使用 Java 8 構建自我可執行文件時,Dataflow 將 userAgent 顯示為 Apache_Beam_SDK_for_Java/2.24.0(JRE_8_environment)
並使用 Java 11 顯示 Apache_Beam_SDK_for_Java/2.24.0(JDK_11_environment)
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.