[英]Solr SimplePostTool: IOException while reading response: java.net.SocketException: Unexpected end of file from server
I'm trying to index a large file repository (10,000+ documents of every format under the sun) using Solr 6.1.0's simpleposttool (bin/post). 我正在尝试使用Solr 6.1.0的simpleposttool(bin / post)对大型文件存储库(10,000种以上各种格式的文档)建立索引。 It'll run for quite a while with no problems then crash with the following: 它会运行一段时间,不会出现任何问题,然后崩溃,并显示以下内容:
POSTing file ThingsGoingWellUpToHere.pdf (application/pdf) to [base]/extract
POSTing file EXAMPLE1.pdf (application/pdf) to [base]/extract
SimplePostTool: WARNING: IOException while reading response: java.net.SocketException: Unexpected end of file from server
POSTing file EXAMPLE2.pptx (application/vnd.openxmlformats-officedocument.presentationml.presentation) to [base]/extract
SimplePostTool: FATAL: Connection error (is Solr running at http://localhost:8983/solr/sample/update ?): java.net.ConnectException: Connection refused
At this point Solr goes down too: 此时,Solr也下降了:
$ solr status
Found 1 Solr nodes:
Solr process 26499 from /opt/solr-6.1.0/bin/solr-8983.pid not found.
I wind up having to solr restart
whenever this happens. 每当出现这种情况时,我都会不得不solr restart
。 Anyone else run into a similar issue? 还有其他人遇到类似的问题吗?
Quick note, if I had to take a wild guess it's something to do with corrupt files. 快速说明,如果我不得不大胆猜测,则与损坏的文件有关。 The collection I'm working with is ~25Gb and has gone through two layers of SCP on spotty connections. 我正在使用的集合约为25Gb,并且在不规则的连接上经过了两层SCP。 If this turns out to be the case I'll close this out myself. 如果真是这样,我自己将其关闭。
EDIT: Tried posting individual documents that SimplePostTool had failed on and they went through fine, so it's unlikely to be a corruption issue. 编辑:尝试过发布SimplePostTool失败的单个文档,并且它们运行良好,因此不太可能是损坏问题。 The search continues... 搜索继续...
It was totally a memory issue. 这完全是一个内存问题。 If you see this error, assume you didn't allocate enough memory to your Solr instance. 如果看到此错误,请假定您没有为Solr实例分配足够的内存。 Just bump this up with the -Xmx
flag when using solr start
. 使用solr start
时,只需使用-Xmx
标志将其solr start
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.