Solr DataImportHandler doesn't work with XML Files

Question

I'm very new to Solr. I succeeded in indexing data from my sql database via DIH. Now I want to import xml files and index them also via DIH but it just won't work! My data-config.xml looks like this:

<dataConfig>
    <dataSource type="FileDataSource" encoding="UTF-8" />
    <document>
    <entity name="dir" 
            processor="FileListEntityProcessor" 
            baseDir="/bla/test2" 
            fileName=".*xml"
            stream="true"
            recursive="false"       
            rootEntity="false">
            <entity name="PubmedArticle"
                    processor="XPathEntityProcessor"
                    transformer="RegexTransformer"
                    stream="true"
                    forEach="/PubmedArticle"
                    url="${dir.fileAbsolutePath}">


                <field column="journal" xpath="//Name[.='journal']/following-sibling::Value/text()" />
                <field column="authors" xpath="//Name[.='authors']/following-sibling::Value/text()" />

             ..etc

And i have the following fields in schema.xml:

<field name="journal" type="text" indexed="true" stored="true" required="true" /> <field name="authors" type="text" indexed="true" stored="true" required="true" />

When i run Solr i get no errors and no document is indexed:

<str name="Total **Rows Fetched**">**2000**</str>
<str name="Total **Documents Skipped**">**0**</str>
<str name="Full Dump Started">2012-02-01 14:59:17</str>
<str name="">Indexing completed. **Added/Updated: 0 documents.** Deleted 0 documents.

Can anyone tell me what i did wrong?! I have even double checked the path syntax...

Answer 1

I recently encountered the same problem when trying the same thing; ie, when using FileListEntityProcessor (to read multiple local .xml files) and XPathEntityProcessor (to grab certain XML elements).

Root Cause : is in this line:

<field column="journal" xpath="//Name[.='journal']/following-sibling::Value/text()" />

Explanation : the argument for the xpath attribute ("//Name..."), while valid xpath syntax, is NOT supported by Solr. The "Apache Solr 4.4 Reference Guide" simply says: The XPath expression which will extract the content from the record for this field. Only a subset of Xpath syntax is supported.

Solution : Change the argument for xpath to be the full path from the document root:

<field column="journal" xpath="/full/path/from/root/of/document/Name[.='journal']/following-sibling::Value/text()" />

Answer 2

I'd suggest reviewing the answers to a similar question:

Need help indexing XML files into Solr using DataImportHandler

Using a scripting language like groovy is a lot less complicated and easier to test.