堆棧內存溢出
  簡體   English   中英

如何在駱駝路線中解決“ java.nio.charset.MalformedInputException：輸入長度= 1”？

[英]How to fix “java.nio.charset.MalformedInputException: Input length = 1” in Camel route?

 原文  2019-09-06 10:52:19  2   1    java/ apache-camel

問題描述

我有一個駱駝路線，該路線會輪詢ftp服務器以獲取zip文件，從zip中提取一個csv文件，然后將csv的每一行作為json正文發送一個http請求。 某些csv文件的大小小於10MB，而某些文件的大小約為500MB。 問題是我無法完全處理較小的文件，發送了一些行，然后出現MalformedInputException並開始回滾。

到目前為止，我發現解決此問題的唯一方法是在拆分之前將convertBodyTo（String.class）修改 。 這樣可以解決較小文件的問題，但是較大文件會導致堆空間錯誤，因為整個文件都已加載到內存中。 

    DataFormat dataFormat = BindyCsvDataFormat(myModel.class);

    from("ftp://" + username + "@" + ftpUrl + "?password=" + password + "&noop=True" +
            "&binary=True&sortBy=reverse:file:modified&antInclude=" + filePrefix + "*" +
            "&eagerMaxMessagesPerPoll=false&maxMessagesPerPoll=1")
            .streamCaching()
            .unmarshal().zipFile()
            .convertBodyTo(String.class) // works for small files with this line, but not for large
            .split(body().tokenize("\r\n", 1, true))
            .streaming()
            .unmarshal(dataFormat)
            .marshal().json(JsonLibrary.XStream)
            .to(outputUrl)
            .end();

Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
org.apache.camel.RuntimeCamelException: Scanner aborted because of an IOException!
    at org.apache.camel.processor.Splitter$SplitterIterable$1.hasNext(Splitter.java:175) ~[camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.processor.MulticastProcessor$MulticastState.run(MulticastProcessor.java:304) ~[camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.impl.engine.DefaultReactiveExecutor$3.run(DefaultReactiveExecutor.java:116) ~[camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:185) [camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:59) [camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:87) [camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:228) [camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:454) [camel-file-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:223) [camel-file-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:186) [camel-file-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.support.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:172) [camel-support-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.support.ScheduledPollConsumer.run(ScheduledPollConsumer.java:99) [camel-support-3.0.0-RC1.jar:3.0.0-RC1]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
    at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.nio.charset.MalformedInputException: Input length = 1
    at java.nio.charset.CoderResult.throwException(CoderResult.java:274) ~[?:?]
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339) ~[?:?]
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) ~[?:?]
    at java.io.InputStreamReader.read(InputStreamReader.java:185) ~[?:?]
    at java.io.Reader.read(Reader.java:189) ~[?:?]
    at org.apache.camel.util.Scanner.readMore(Scanner.java:162) ~[camel-util-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.util.Scanner.next(Scanner.java:136) ~[camel-util-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.util.Scanner.next(Scanner.java:44) ~[camel-util-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.support.GroupTokenIterator.doNext(GroupTokenIterator.java:122) ~[camel-support-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.support.GroupTokenIterator.next(GroupTokenIterator.java:112) ~[camel-support-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.processor.Splitter$SplitterIterable$1.next(Splitter.java:182) ~[camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.processor.Splitter$SplitterIterable$1.next(Splitter.java:158) ~[camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.processor.MulticastProcessor$MulticastState.run(MulticastProcessor.java:303) ~[camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    ... 16 more

java.nio.charset.MalformedInputException: Input length = 1
    at java.nio.charset.CoderResult.throwException(CoderResult.java:274) ~[?:?]
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339) ~[?:?]
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) ~[?:?]
    at java.io.InputStreamReader.read(InputStreamReader.java:185) ~[?:?]
    at java.io.Reader.read(Reader.java:189) ~[?:?]
    at org.apache.camel.util.Scanner.readMore(Scanner.java:162) ~[camel-util-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.util.Scanner.next(Scanner.java:136) ~[camel-util-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.util.Scanner.next(Scanner.java:44) ~[camel-util-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.support.GroupTokenIterator.doNext(GroupTokenIterator.java:122) ~[camel-support-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.support.GroupTokenIterator.next(GroupTokenIterator.java:112) ~[camel-support-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.processor.Splitter$SplitterIterable$1.next(Splitter.java:182) ~[camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.processor.Splitter$SplitterIterable$1.next(Splitter.java:158) ~[camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.processor.MulticastProcessor$MulticastState.run(MulticastProcessor.java:303) [camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.impl.engine.DefaultReactiveExecutor$3.run(DefaultReactiveExecutor.java:116) [camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:185) [camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:59) [camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:87) [camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:228) [camel-base-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:454) [camel-file-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:223) [camel-file-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:186) [camel-file-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.support.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:172) [camel-support-3.0.0-RC1.jar:3.0.0-RC1]
    at org.apache.camel.support.ScheduledPollConsumer.run(ScheduledPollConsumer.java:99) [camel-support-3.0.0-RC1.jar:3.0.0-RC1]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
    at java.lang.Thread.run(Thread.java:834) [?:?]```

1 個解決方案

解決方案1
 0  2019-09-09 15:37:07

該錯誤信息指出了編碼問題。 兩種不同類型的文件的編碼是否不同 ？

如果是，則必須以某種方式確定文件的編碼方式。 或者至少知道您獲得的不同類型，然后為文件分配正確的編碼。

如果您知道編碼，則可以按照@bedla的建議將其設置為Exchange屬性。

因此，如果小文件是例如ISO編碼的，則可以嘗試 

.setProperty(Exchange.CHARSET_NAME, constant("ISO-8859-1"))

暫無
暫無

聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

相關問題 java.nio.charset.MalformedInputException：輸入長度 java.nio.charset.MalformedInputException：輸入長度= 1 java.nio.charset.MalformedInputException：輸入長度= 1 為什么會出錯？ (java.nio.charset.MalformedInputException: 輸入長度 = 1) 全包字符集以避免“java.nio.charset.MalformedInputException：輸入長度= 1”？ java.io.UncheckedIOException: java.nio.charset.MalformedInputException: 輸入長度 = 1 Gradle構建失敗：錯誤：任務'：app：processDebugResources'的執行失敗。 > java.nio.charset.MalformedInputException：輸入長度= 1 讀取CSV檔案時出現java.nio.charset.MalformedInputException Source.fromResource（）在本地工作，但在服務器上拋出java.nio.charset.MalformedInputException Jenkins java.nio.charset.UnmappableCharacterException：輸入長度 = 1
相關標簽
 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM