如何使用Solr将文件发送到HDFS

Question

I am trying to index files and keep these files on HDFS.

I am using Solr 4.10.4 and Hadoop 2.6.0.

The command I use for integration HDFS and Solr is;

java -Dsolr.directoryFactory=HdfsDirectoryFactory -Dsolr.lock.type=hdfs -Dsolr.data.dir=hdfs://<hostname>:<port>/Solr -Dsolr.updatelog=hdfs://<hostname>:<port>/Solr-logs -jar start.jar

When I try to send some files using

java -jar post.jar <file-path>

command, Solr gives warning and does not send it to hdfs. It only accepts xml format with specific fields.

How can I send any kind of file using Solr for indexing to HDFS and do search on these files using Solr.

Note: I am beginner on Solr and Hadoop.

Answer 1

if want to index other file format you can specify document type as like below

java -Dtype=text/csv -jar post.jar *.csv

By default below command takes care of all possible file types that Apache Tika can handle

java -Dauto -jar post.jar *.*