使用自定義篩選器篩選Accumulo返回的結果時出錯

Question

我寫了一個非常簡單的自定義過濾器來過濾Accumulo返回的結果。 這是我寫的過濾器

public class MyFilter extends Filter {

    @Override
    public boolean accept(Key key, Value val) {
        Long page = 1L;
        Integer limit = 25;

        if(key.getColumnQualifier().getBytes().equals("Class".getBytes()) && val.get().equals("1".getBytes())) {
            if(page == 1) {
                return true;
            }
            limit--;
            if(limit == 1L) {
                page++;
                limit = 25;
            }
        }

        return false;
    }
}

我把這個過濾器加到了Accumulo的掃描儀上

Set<Range> ranges = new HashSet<>();
IteratorSetting iter = new IteratorSetting(15, "MyFilter", MyFilter.class);
myScanner.addScanIterator(iter);
Iterator<Entry<Key, Value>> kys = myScanner.iterator();
while(kys.hasNext()) { // This is line 335 in com.latize.ulysses3.service.AccumuloPivotTable.getRows
    Entry<Key, Value> e = kys.next();
    ranges.add(Range.exact(e.getKey().getRow()));
}

但每當我嘗試運行這段代碼時，我都會得到這個堆棧跟蹤

java.lang.RuntimeException: org.apache.accumulo.core.client.impl.AccumuloServerException: Error on server accumulo.tablet.2:9997
        at org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:161)
        at com.latize.ulysses3.service.AccumuloPivotTable.getRows(AccumuloPivotTable.java:335)
        at com.latize.ulysses3.webservice.DataVault.getFilteredRows(DataVault.java:950)
        at com.latize.ulysses3.webservice.DataVault$Proxy$_$$_WeldClientProxy.getFilteredRows(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
        at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
        at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
        at org.jboss.resteasy.core.SynchronousDispatcher.invokePropagateNotFound(SynchronousDispatcher.java:217)
        at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:224)
        at org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.doFilter(FilterDispatcher.java:62)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.marmotta.platform.core.servlet.MarmottaResourceFilter$LMFFilterChain.doFilter(MarmottaResourceFilter.java:189)
        at org.apache.marmotta.platform.core.filters.ModuleResourceFilter.doFilter(ModuleResourceFilter.java:169)
        at org.apache.marmotta.platform.core.filters.ModuleResourceFilter$Proxy$_$$_WeldClientProxy.doFilter(Unknown Source)
        at org.apache.marmotta.platform.core.servlet.MarmottaResourceFilter$LMFFilterChain.doFilter(MarmottaResourceFilter.java:184)
        at com.latize.ulysses.platform.core.filters.ClickJackFilter.doFilter(ClickJackFilter.java:105)
        at com.latize.ulysses.platform.core.filters.ClickJackFilter$Proxy$_$$_WeldClientProxy.doFilter(Unknown Source)
        at org.apache.marmotta.platform.core.servlet.MarmottaResourceFilter$LMFFilterChain.doFilter(MarmottaResourceFilter.java:184)
        at com.latize.ulysses.platform.core.filters.SessionTimeoutCookieFilter.doFilter(SessionTimeoutCookieFilter.java:125)
        at com.latize.ulysses.platform.core.filters.SessionTimeoutCookieFilter$Proxy$_$$_WeldClientProxy.doFilter(Unknown Source)
        at org.apache.marmotta.platform.core.servlet.MarmottaResourceFilter$LMFFilterChain.doFilter(MarmottaResourceFilter.java:184)
        at org.apache.marmotta.platform.core.filters.MarmottaServerNameFilter.doFilter(MarmottaServerNameFilter.java:104)
        at org.apache.marmotta.platform.core.servlet.MarmottaResourceFilter$LMFFilterChain.doFilter(MarmottaResourceFilter.java:184)
        at com.latize.ulysses.platform.core.filters.UlyssesTemplatingFilter.doFilter(UlyssesTemplatingFilter.java:178)
        at com.latize.ulysses.platform.core.filters.UlyssesTemplatingFilter$Proxy$_$$_WeldClientProxy.doFilter(Unknown Source)
        at org.apache.marmotta.platform.core.servlet.MarmottaResourceFilter$LMFFilterChain.doFilter(MarmottaResourceFilter.java:184)
        at org.apache.marmotta.platform.core.filters.TemplatingFilter.doFilter(TemplatingFilter.java:176)
        at org.apache.marmotta.platform.core.filters.TemplatingFilter$Proxy$_$$_WeldClientProxy.doFilter(Unknown Source)
        at org.apache.marmotta.platform.core.servlet.MarmottaResourceFilter$LMFFilterChain.doFilter(MarmottaResourceFilter.java:184)
        at org.apache.marmotta.platform.security.filters.MarmottaAccessControlFilter.doFilter(MarmottaAccessControlFilter.java:142)
        at org.apache.marmotta.platform.security.filters.MarmottaAccessControlFilter$Proxy$_$$_WeldClientProxy.doFilter(Unknown Source)
        at org.apache.marmotta.platform.core.servlet.MarmottaResourceFilter$LMFFilterChain.doFilter(MarmottaResourceFilter.java:184)
        at org.apache.marmotta.platform.user.filters.MarmottaAuthenticationFilter.doFilter(MarmottaAuthenticationFilter.java:228)
        at org.apache.marmotta.platform.user.filters.MarmottaAuthenticationFilter$Proxy$_$$_WeldClientProxy.doFilter(Unknown Source)
        at org.apache.marmotta.platform.core.servlet.MarmottaResourceFilter$LMFFilterChain.doFilter(MarmottaResourceFilter.java:184)
        at org.apache.marmotta.platform.core.servlet.MarmottaResourceFilter.doFilter(MarmottaResourceFilter.java:135)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.marmotta.platform.core.servlet.MarmottaPreStartupFilter.doFilter(MarmottaPreStartupFilter.java:106)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:169)
        at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:232)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.marmotta.platform.core.servlet.MarmottaOptionsFilter.doFilter(MarmottaOptionsFilter.java:83)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
        at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:318)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.accumulo.core.client.impl.AccumuloServerException: Error on server accumulo.tablet.2:9997
        at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:293)
        at org.apache.accumulo.core.client.impl.ScannerIterator$Reader.run(ScannerIterator.java:80)
        at org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:151)
        ... 69 more
Caused by: org.apache.thrift.TApplicationException: Internal error processing startScan
        at org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
        at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.recv_startScan(TabletClientService.java:232)
        at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.startScan(TabletClientService.java:208)
        at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:410)
        at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:285)
        ... 71 more

accumulo.tablet.2：9997是運行我的一台平板電腦服務器的機器的域名。 配置很好，因為其他功能正常工作。 港口也在上升。 有人能告訴我為什么我會收到這個錯誤？ 任何幫助，將不勝感激。

Answer 1

要正確使用過濾器，必須將它們添加到accumulo類路徑中，通常是通過部署.jar文件，其中在$ACCUMULO_HOME/lib或$ACCUMULO_HOME/lib/ext系統的每個節點上定義迭代器。 為了更深入地理解迭代器，我建議使用accumulo的非常好的文檔 。

為了便於部署，我想建議使用Jenkins ， Puppet或Chef 。 如果這對您的項目來說是一個很大的努力，那么一個簡單的shell腳本也可以。 您可能會發現群集shell cssh對管理問題也非常有幫助！ 請注意，您不需要 accumulo的任何重新啟動在部署后ext文件夾，但它是必要的lib文件夾。

Answer 2

我遇到了這個問題。 當我檢查tserver的日志時，我得到了一個ClassNotFound異常。 我似乎需要將該類添加到Accumulo的類路徑中。 但是，如果我想將這些過濾結果作為應用程序的一部分而不是從終端運行它，我將如何去做呢？ 我可能無法將過濾器添加到Accumulo的類路徑中。