Solr挂起在Linux OS上创建内核的原因是什么？

Question

Can anyone help identify the cause of Solr hanging when it tries to create a core. This happens on a Red Hat Linux machine when Solr tries to create the core on startup and when it is specifically requested to create a core. It does create the data directory for the Lucene index and 'write.lock' file but then the process halts. Solr is still responsive but the core is stuck in the loading phase.

I've picked this out from the thread dump:

coreLoadExecutor-6-thread-1 (25)
sun.nio.fs.UnixNativeDispatcher.stat0​(Native Method)
sun.nio.fs.UnixNativeDispatcher.stat​(UnixNativeDispatcher.java:286)
sun.nio.fs.UnixFileAttributes.get​(UnixFileAttributes.java:70)
sun.nio.fs.UnixFileStore.devFor​(UnixFileStore.java:55)
sun.nio.fs.UnixFileStore.<init>​(UnixFileStore.java:70)
sun.nio.fs.LinuxFileStore.<init>​(LinuxFileStore.java:48)
sun.nio.fs.LinuxFileSystem.getFileStore​(LinuxFileSystem.java:112)
sun.nio.fs.UnixFileSystem$FileStoreIterator.readNext(UnixFileSystem.java:213)
sun.nio.fs.UnixFileSystem$FileStoreIterator.hasNext(UnixFileSystem.java:224)
org.apache.lucene.util.IOUtils.getFileStore​(IOUtils.java:543)
org.apache.lucene.util.IOUtils.spinsLinux​(IOUtils.java:487)
org.apache.lucene.util.IOUtils.spins​(IOUtils.java:476)
org.apache.lucene.util.IOUtils.spins​(IOUtils.java:451)
org.apache.lucene.index.ConcurrentMergeScheduler.initDynamicDefaults(ConcurrentMergeScheduler.java:376)
org.apache.lucene.index.ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:464)
org.apache.lucene.index.IndexWriter.waitForMerges(IndexWriter.java:2425)
org.apache.lucene.index.IndexWriter.shutdown​(IndexWriter.java:1118)
org.apache.lucene.index.IndexWriter.close​(IndexWriter.java:1162)
org.apache.solr.update.SolrIndexWriter.close​(SolrIndexWriter.java:142)
org.apache.solr.core.SolrCore.initIndex​(SolrCore.java:588)
org.apache.solr.core.SolrCore.<init>​(SolrCore.java:762)
org.apache.solr.core.SolrCore.<init>​(SolrCore.java:688)
org.apache.solr.core.CoreContainer.create​(CoreContainer.java:838)
org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:494)
org.apache.solr.core.CoreContainer$$Lambda$25/1297978429.call​(Unknown Source)
java.util.concurrent.FutureTask.run​(FutureTask.java:266)
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0​(ExecutorUtil.java:229)
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$26/348984985.run​(Unknown Source)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run​(Thread.java:745)

This is the longest running thread along with, rather ominously, 'DestroyJavaVM (28)'

I have checked the permissions and tried various indexing configurations in the solrconfig. The same install / configuration of Solr works fine on other machines. I think this is most likely file system or OS related.

Answer 1

So this did turn out to be a bad disk of some sort. I still haven't got to the bottom of what was wrong with it. On the face of it there was space on the drive and no obvious problems with it. Changing the Solr home to another mounted drive fixed the problem. I've never seen this behavior before, the stat call never came back even after being left for several days.

So if you have this problem and any alternative drives are available - it can't hurt to give them a try.