How to filter out dot and dot dot(default unix inodes) while polling a directory using spring integration

Question

I am trying to use spring integration to poll some of my directories. Since spring integration writes in KeyValue table and it picks up this . as well as .. unix inode and tries to write repeatedly to my db table it throws exception after a few seconds of trying. I would like to learn a way to ignore them for spring to pickup. I am already using AcceptOnceFileListFilter. Also I have tried with a custom filter that filters out any file with filename length less than 3 but they are not working.

Update:

I am using IgnoreHi‌​ddenFileListFilter yet its not working. I have narrowed the issue that Since my custom filter implements FileListFilter < File > all the files coming to this filter are java.io.File objects and it does not recognize . and .. as Files. I confirmed this since my sftp remote filter implements FileListFilter< LsEntry > and it is correctly filtering out . and .. which i can see in logs. However i cannot use LsEntry in local as it throws a type cast exception.

Here is my configuration for filters:

<bean id="remoteCompositeFilter" class="org.springframework.integration.file.filters.CompositeFileListFilter">
<constructor-arg>
  <list>
    <bean class="com.file.RemoteCustomFilter"/>
  <bean class="org.springframework.integration.file.filters.AcceptOnceFileListFilter" />
    <!-- download files whose extension is not .writing -->
    <bean class="org.springframework.integration.sftp.filters.SftpRegexPatternFileListFilter" >
      <constructor-arg value="^.*(?&lt;!.writing)$" /> 
    </bean>
    <!-- List of qualifying files to be retrieved from remote server are stored in KEY_VALUE_STORE table.
         jdbcMetadataStore is defined in test-sftp-common.xml -->
    <bean id="remotePersistentFilter" 
        class="org.springframework.integration.sftp.filters.SftpPersistentAcceptOnceFileListFilter">
        <constructor-arg name="store" ref="jdbcMetadataStore"/>
        <constructor-arg name="prefix" value="" />
        <property name="flushOnUpdate" value="true" />
    </bean>
  </list>
</constructor-arg>

<bean id="localCompositeFilter" class="org.springframework.integration.file.filters.CompositeFileListFilter">
<constructor-arg>
  <list>
    <bean class="com.file.CustomFilter"/>
    <bean class="org.springframework.integration.file.filters.AcceptOnceFileListFilter" />
    <bean class="org.springframework.integration.file.filters.IgnoreHiddenFileListFilter" />

    <!-- download files whose extension is not .writing -->
    <bean class="org.springframework.integration.file.filters.RegexPatternFileListFilter" >
        <constructor-arg value="^.*(?&lt;!.writing)$" /> 
    </bean>
    <bean id="localIncomingPersistentFilter"
        class="org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter">
        <constructor-arg name="store" ref="propertiesMetadataStore"/>
        <constructor-arg name="prefix" value="" />
        <property name="flushOnUpdate" value="true" />
    </bean>
  </list>
</constructor-arg>

And here is the CustomFilter

public class CustomFilter implements FileListFilter<File> {
private static Logger log = Logger.getLogger(CustomFilter.class); 
@Override
public List<File> filterFiles(File[] files) {
    List<File> filteredFileList = new LinkedList<File>();
    for (File this_file : files){
        log.info("Checking file : " + this_file.getAbsolutePath());
        if(!this_file.getName().startsWith(".")){
            filteredFileList.add(this_file);
            log.info("Adding local file to filtered list : " + this_file.getAbsolutePath());
        }
        else{
            log.info("Rejecting local file from filtered list : " + this_file.getAbsolutePath());
        }
    }
    return filteredFileList;
}

}

RemoteCustomFilter

public class RemoteCustomFilter implements FileListFilter<LsEntry> {
private static Logger log = Logger.getLogger(CustomFilter.class); 
@Override
public List<LsEntry> filterFiles(LsEntry[] files) {
    List<LsEntry> filteredFileList = new LinkedList<LsEntry>();
    for (LsEntry this_file : files){
        log.info("Checking file : " + this_file.getFilename());
        if(!this_file.getFilename().startsWith(".")){
            filteredFileList.add(this_file);
            log.info("Adding remote file to filtered list : " + this_file.getFilename());
        }
        else{
            log.info("Rejecting remote file from filtered list : " + this_file.getFilename());
        }
    }
    return filteredFileList;
}

}

This is what my DEBUG log looks like:

2017-02-02|13:26:17.878 DEBUG [task-scheduler-3] org.springframework.integration.util.SimplePool [doGetItem:190] - Obtained new org.springframework.integration.sftp.session.SftpSession@3221346d.
2017-02-02|13:26:18.423 INFO  [task-scheduler-1] com.file.CustomFilter [filterFiles:16] - Checking file : /outgoing/.DS_Store
2017-02-02|13:26:18.423 INFO  [task-scheduler-1] com.file.CustomFilter [filterFiles:22] - Rejecting local file from filtered list : /outgoing/.DS_Store
2017-02-02|13:26:18.424 DEBUG [task-scheduler-1] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:261] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.021 DEBUG [task-scheduler-4] org.springframework.integration.util.SimplePool [doGetItem:190] - Obtained new org.springframework.integration.sftp.session.SftpSession@3b560425.
2017-02-02|13:26:19.095 DEBUG [task-scheduler-3] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.095 DEBUG [task-scheduler-2] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.095 DEBUG [task-scheduler-1] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.095 DEBUG [task-scheduler-7] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.095 DEBUG [task-scheduler-9] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.095 DEBUG [task-scheduler-10] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.095 DEBUG [task-scheduler-6] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.095 DEBUG [task-scheduler-2] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.095 DEBUG [task-scheduler-7] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.095 DEBUG [task-scheduler-3] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.095 DEBUG [task-scheduler-9] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.095 DEBUG [task-scheduler-10] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:208] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:19.372 INFO  [task-scheduler-3] com.file.CustomFilter [filterFiles:17] - Checking file : .
2017-02-02|13:26:19.372 INFO  [task-scheduler-3] com.file.CustomFilter [filterFiles:23] - Rejecting remote file from filtered list : .
2017-02-02|13:26:19.372 INFO  [task-scheduler-3] com.file.CustomFilter [filterFiles:17] - Checking file : ..
2017-02-02|13:26:19.372 INFO  [task-scheduler-3] com.file.CustomFilter [filterFiles:23] - Rejecting remote file from filtered list : ..
2017-02-02|13:26:19.854 DEBUG [task-scheduler-3] org.springframework.integration.file.remote.session.CachingSessionFactory [close:187] - Releasing Session org.springframework.integration.sftp.session.SftpSession@3221346d back to the pool.
2017-02-02|13:26:19.854 DEBUG [task-scheduler-3] org.springframework.integration.util.SimplePool [releaseItem:221] - Releasing org.springframework.integration.sftp.session.SftpSession@3221346d back to the pool
2017-02-02|13:26:19.854 DEBUG [task-scheduler-3] org.springframework.integration.sftp.inbound.SftpInboundFileSynchronizer [synchronizeToLocalDirectory:270] - 0 files transferred
2017-02-02|13:26:19.854 INFO  [task-scheduler-3] com.file.CustomFilter [filterFiles:16] - Checking file : incoming/.DS_Store
2017-02-02|13:26:19.855 INFO  [task-scheduler-3] com.file.CustomFilter [filterFiles:22] - Rejecting local file from filtered list : incoming/.DS_Store
2017-02-02|13:26:19.855 DEBUG [task-scheduler-3] org.springframework.integration.endpoint.SourcePollingChannelAdapter [doPoll:261] - Received no Message during the poll, returning 'false'
2017-02-02|13:26:21.080 INFO  [task-scheduler-1] com.file.CustomFilter [filterFiles:16] - Checking file : incoming/.DS_Store
2017-02-02|13:26:21.080 INFO  [task-scheduler-1] com.file.CustomFilter [filterFiles:22] - Rejecting local file from filtered list : incoming/.DS_Store
2017-02-02|13:26:21.081 DEBUG [task-scheduler-1] org.springframework.integration.util.SimplePool [doGetItem:185] - Obtained org.springframework.integration.sftp.session.SftpSession@3221346d from pool.
2017-02-02|13:26:21.101 INFO  [task-scheduler-4] com.file.CustomFilter [filterFiles:17] - Checking file : .
2017-02-02|13:26:21.102 INFO  [task-scheduler-4] com.file.CustomFilter [filterFiles:23] - Rejecting remote file from filtered list : .
2017-02-02|13:26:21.102 INFO  [task-scheduler-4] com.file.CustomFilter [filterFiles:17] - Checking file : ..
2017-02-02|13:26:21.102 INFO  [task-scheduler-4] com.file.CustomFilter [filterFiles:23] - Rejecting remote file from filtered list : ..
2017-02-02|13:26:21.707 DEBUG [task-scheduler-4] org.springframework.integration.file.remote.session.CachingSessionFactory [close:187] - Releasing Session org.springframework.integration.sftp.session.SftpSession@3b560425 back to the pool.
2017-02-02|13:26:21.707 DEBUG [task-scheduler-4] org.springframework.integration.util.SimplePool [releaseItem:221] - Releasing org.springframework.integration.sftp.session.SftpSession@3b560425 back to the pool
2017-02-02|13:26:21.707 DEBUG [task-scheduler-4] org.springframework.integration.sftp.inbound.SftpInboundFileSynchronizer [synchronizeToLocalDirectory:270] - 0 files transferred

But here is my KeyValue Table View of my table

Answer 1

Use the IgnoreHiddenFileListFilter

or a regex filter with [^.].* .