[英]Using Pentaho Kettle, how do I automatically retry rest requests which fail due to connection hiccups?
我們有一個 Pentaho BI 系統,它在眾多數據源中查詢特定 REST api,每次運行超過 20k 查詢變體。
可以預見,在大多數運行中,這些請求中的一些會由於連接問題而失敗。 這些通常在 Pentaho 日志中表現為org.pentaho.di.core.exception.KettleException
由於javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
,最終由java.io.EOFException: SSL peer shut down incorrectly
引起java.io.EOFException: SSL peer shut down incorrectly
。
在搜索互聯網和 Pentaho 論壇后,我們找不到任何關於在這種情況下添加簡單重試方法的說明。 頭發正在被扯掉。
更新 1:堆棧跟蹤:
ERROR 29-11 11:02:17,659 - B - org.pentaho.di.core.exception.KettleException:
Can not result from [https://<DOMAIN>/<PATH>?<PARAMS>]
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at org.pentaho.di.trans.steps.rest.Rest.callRest(Rest.java:190)
at org.pentaho.di.trans.steps.rest.Rest.processRow(Rest.java:385)
at org.pentaho.di.trans.step.RunThread.run(RunThread.java:40)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.sun.jersey.api.client.ClientHandlerException: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at com.sun.jersey.client.apache.DefaultApacheHttpMethodExecutor.executeMethod(DefaultApacheHttpMethodExecutor.java:213)
at com.sun.jersey.client.apache.ApacheHttpClientHandler.handle(ApacheHttpClientHandler.java:175)
at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:81)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.get(WebResource.java:191)
at org.pentaho.di.trans.steps.rest.Rest.callRest(Rest.java:141)
... 3 more
Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:817)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1138)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:632)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at org.apache.commons.httpclient.HttpConnection.flushRequestOutputStream(HttpConnection.java:827)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.flushRequestOutputStream(MultiThreadedHttpConnectionManager.java:1525)
at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:1975)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:993)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
at com.sun.jersey.client.apache.DefaultApacheHttpMethodExecutor.executeMethod(DefaultApacheHttpMethodExecutor.java:210)
... 9 more
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)
... 22 more
或者在其余客戶端步驟上使用步驟錯誤處理,並將錯誤字段定向到另一個其余客戶端步驟。
顯然,這只能讓您根據步驟重試多少次,但無論如何重試一次都沒有意義。
使用來源,盧克。
在 org.pentaho.di.trans.steps.rest.Rest.processRow(Rest.java:385) 你會發現
} catch(KettleException e) {
boolean sendToErrorRow=false;
String errorMessage = null;
if (getStepMeta().isDoingErrorHandling()){
sendToErrorRow = true;
errorMessage = e.toString();
} else {
logError(BaseMessages.getString(PKG, "Rest.ErrorInStepRunning")+e.getMessage()); //$NON-NLS-1$
setErrors(1);
logError(Const.getStackTracker(e));
stopAll();
setOutputDone(); // signal end to receiver(s)
return false;
}
if (sendToErrorRow) {
// Simply add this row to the error row
putError(getInputRowMeta(), r, 1, errorMessage, null, "Rest001");
}
}
看來您可以按行配置錯誤處理。 這似乎就是你所追求的。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.