SOLR mysql DataImport missing mandatory uniqueKey field: id

Question

I'm new to SOLR. Using DataImporter to bring MySQL records in for indexing. Import works, but 0 Documents get created. I'm seeing the following error:

org.apache.solr.common.SolrException: Document is missing mandatory uniqueKey field: id

My data-config.xml looks as follows:

<dataConfig>  
<dataSource type="JdbcDataSource" driver="org.gjt.mm.mysql.Driver" url="jdbc:mysql://localhost/gartner" user="" password="" batchSize="-1" />  
<document name="inquiries">
    <entity name="inquiry_data" query="SELECT * FROM research_srv_req">
        <field column="reference_id" name="reference_id" />
        <field column="type" name="type" />
        <field column="sub_type" name="sub_type" />
        <field column="subject" name="inquiry_subject" />
        <field column="advice" name="advice" />
        <field column="client_question" name="client_question" />
        <field column="decision_stage_desc" name="decision_stage_desc" />
        <field column="acct_name" name="acct_name" />
        <field column="acct_sic_type" name="acct_sic_type" />
        <field column="acct_sic_sub_type" name="acct_sic_sub_type" />
        <field column="enterprise_sic_type" name="enterprise_sic_type" />
        <field column="enterprise_sic_sub_type" name="enterprise_sic_sub_type" />
        <field column="client_job_title" name="client_job_title" />
    </entity>  
</document>  

and I added the following to my schema.xml file:

<field name="reference_id" type="int" indexed="true" stored="true" required="true" multiValued="false" />
<field name="type" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="sub_type" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="inquiry_subject" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="advice" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="client_question" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="decision_stage" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="acct_name" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="acct_sic_type" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="acct_sic_sub_type" type="string" indexed="true" stored="true" required="true" multiValued="false" />   
<field name="enterprise_sic_type" type="string" indexed="true" stored="true" required="true" multiValued="false" /> 
<field name="enterprise_sic_sub_type" type="string" indexed="true" stored="true" required="true" multiValued="false" /> 
<field name="client_job_title" type="string" indexed="true" stored="true" required="true" multiValued="false" />

The original schema.xml also had the following line:

<uniqueKey>id</uniqueKey>

I tried changing id to reference_id which is the unique column in my database, but then SOLR fails to load with the following stack dump:

message {msg=SolrCore 'collection1' is not available due to init failure: Error initializing QueryElevationComponent.,trace=org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Error initializing QueryElevationComponent. at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:860) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:251) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor. java:859) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Unknown Source) Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent. at org.apache.solr.core.SolrCore.(SolrCore.java:835) at org.apache.solr.core.SolrCore.(SolrCore.java:629) at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:622) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:657) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:364) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:356) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 more Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent. at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:240) at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:601) at org.apache.solr.core.SolrCore.(SolrCore.java:830) ... 13 more Caused by: java.lang.NumberFormatException: For input string: "MA147LL/A" at java.lang.NumberFormatException.forInputString(Unknown Source) at java.lang.Integer.parseInt(Unknown Source) at java.lang.Integer.parseInt(Unknown Source) at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:387) at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:378) at org.apache.solr.handler.component.QueryElevationComponent$ElevationObj.(QueryElevationComponent.java:136) at org.apache.solr.handler.component.QueryElevationComponent.loadElevationMap(QueryElevationComponent.java:309) at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:223) ... 15 more ,code=500} description The server encou ntered an internal error ({msg=SolrCore 'collection1' is not available due to init failure: Error initializing QueryElevationComponent.,trace=org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Error initializing QueryElevationComponent. at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:860) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:251) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor. java:859) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Unknown Source) Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent. at org.apache.solr.core.SolrCore.(SolrCore.java:835) at org.apache.solr.core.SolrCore.(SolrCore.java:629) at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:622) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:657) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:364) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:356) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 more Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent. at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:240) at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:601) at org.apache.solr.core.SolrCore.(SolrCore.java:830) ... 13 more Caused by: java.lang.NumberFormatException: For input string: "MA147LL/A" at java.lang.NumberFormatException.forInputString(Unknown Source) at java.lang.Integer.parseInt(Unknown Source) at java.lang.Integer.parseInt(Unknown Source) at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:387) at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:378) at org.apache.solr.handler.component.QueryElevationComponent$ElevationObj.(QueryElevationComponent.java:136) at org.apache.solr.handler.component.QueryElevationComponent.loadElevationMap(QueryElevationComponent.java:309) at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:223) ... 15 more ,code=500}) that prevented it from fulf illing this request.

What am I doing wrong?

Answer 1

UniqueKey needs to be a string. You tried to make it an int. Try changing your type for reference_id to string and reindex.

Answer 2

Before going in your schema.xml remove required="true" attribute in the fields

You have two solution

1) In your data-config.xml give any field name as id which is Unique in table. I guess in your table reference_id is Unique

 <field column="reference_id" name="reference_id" />

change name attribute for that field to id like below

 <field column="reference_id" name="id" />

Now go to your schema.xml and search for this key word field name="id" if it is present then check/add attribute required="true"(which means your forcing id is required ) and also remove required=true for other fields.

if the key word field name="id" is not present then create a new one like below

<field name="id" type="string" indexed="true" stored="true" required="true"></field>

Here you go and re-start and run

2)find the keyWord field name="id" , if it is present , then remove that field itself , also remove id tag

Here you go and re-start and run

Hope any one method helps