Can I use SpringBatch to transfer file(video/txt) present of http and read it and write it on my pc?

Question

Right now i am trying to write an http file(sample:"https://www.w3.org/TR/PNG/iso_8859-1.txt") to my local pc using springBatch. But unable to read the file. My Reader file is as follows:

public FlatFileItemReader<String> reader(){
   FlatFileItemReader<String> reader = new FlatFileItemReader<String>();
   reader.setResource(new UrlResource("https://www.w3.org/TR/PNG/iso_8859-1.txt"));
   reader.setRecordSeparatorPolicy(new RecordSeparatorPolicy() {
            @Override
            public boolean isEndOfRecord(String s) {
                if (s.length() == 100)
                    return true;
                return false;
            }

            @Override
            public String postProcess(String s) {
                return s;
            }

            @Override
            public String preProcess(String s) {
                return s;
            }
        });
   return reader;
}

I am reading 100 length chunks from the input and passing it to writer. But i am getting this error:

org.springframework.batch.item.file.FlatFileParseException: Unexpected end of file before record complete
    at org.springframework.batch.item.file.FlatFileItemReader.applyRecordSeparatorPolicy(FlatFileItemReader.java:303) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
    at org.springframework.batch.item.file.FlatFileItemReader.readLine(FlatFileItemReader.java:220) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
    at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:178) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
...

How to read complete .txt file from http and process and write it to desire location?

Also what if instead of txt file, I will pass video file to reader(What should be FlatFileItemReader< ??? > type in this case?

Answer 1

You don't need that custom record separator policy, you can use a PassThroughLineMapper instead.

Also what if instead of txt file, I will pass video file to reader(What should be FlatFileItemReader type in this case?

You can use byte[] as item type or use the SimpleBinaryBufferedReaderFactory .