Apache Solr无法索引JSON文件

Question

I am new to apache Solr , Did some research and learned how to do indexing . Currently i am facing issue with JSON file indexing.

i am unable to do indexing the below mentioned JSON file data format. After removing the "documents" array i am able to do . Not sure why its happening.

I haven't added any configuration schema.xml file . Tried with some samples which are in apache solr those i am able to do indexing.

And also what is the use of id in schema.xml file ? if my json contains the employid can i add employid instead of "id"

[{
  "employid": "E64492",
  "employGroup": "ABC ABC GROUP",
  "ssn": "BE0003565737",
  "country": {
      "countryId": "56",
      "countryName": "india"
  },
  "sector": {
      "sId": "40",
      "sName": "name"
  },
  "documents": [{
      "language": "EN",
      "fileName": "Helloworld.pdf",
      "fileExists": true,
      "employid": "E64492"
  }],
}]

Can some one please help.

Excpetion details:

"org.apache.solr.common.SolrException"],"msg":"Error parsing JSON field value. Unexpected OBJECT_START at [227], field=documents","code":400}}

Answer 1

The issue is explained in the Solr Reference guide section on indexing with JSON , but it is a bit hard to see among all the text.

There are basically two ways to deal with JSON:

1. Solr input format where you specify the fields and recursive structures directly using Solr convention. In this format, you can feed multiple JSON objects to the parser as you are explicit about each object's structure.
2. Generic JSON format that gets mapped to Solr document following the rules you specify (or that are specified by default in the solrconfig.xml for your collection)

The array syntax you used is for the first option - Solr input format. However, that format does not support nested documents in the way the rest of your object is structured, it needs a _childDocuments_ array instead.

And the generic JSON parser can only take one object.

So, you are at the cross-roads and need to decide what you want to do. This, most likely, imply thinking about the schema you want to end-up with and whether you want to define it explicitly or via the mapping rules.

Answer 2

You have to define a schema for corresponding to the document that you are trying to insert.
Also, you have an extra , after documents

"documents": [{
  "language": "EN",
  "fileName": "Helloworld.pdf",
  "fileExists": true,
  "employid": "E64492"
}],

Regarding the id field, you can rename it to employe_id but also remember to change the tag <uniqueKey>id</uniqueKey> to employe_id

You can also have a schema without unique_key . Check this for more information on unique keys.

Answer 3

Maddy what you are trying to index is a Nested JSON object !!! Solr only allows JSON data to be indexed in FLAT format . By that I mean, Country and Sector object cannot be indexed in the way that you are trying to . You have to flatten them as separate fields, ie Country.countryId must be one separate field, Country.countryName must be a separate field . Similarly , Sector.sId must be a separate field , and Sector.sectorName must be a separate field . Also the objects inside the last document JSON object should be declared in same fashion as employee Id is declared, you need to remove document object and put every field freely . I hope you get the point . This will 100℅ work . I repeat , you cannot index a Nested JSON like this, you need to flatten the JSON to the simplest . Let me know if that helps :) . To more understand the point beneath, on Solr admin screen , take this JSON and try to index it in the Documents section, while keep the network tab open in Chrome or some other browser by clicking F12, you will see the same error that you are getting in the Console !! That is the reason that while you can keep Country and Sector objects as same, but you need to remove data Objects and declare fields inside it freely .

Answer 4

Finally I am able to do indexing after adding the schema definition specified below

     <field name="buyLimit" type="tdoubles"/>
      <field name="country.countryId" type="tlongs"/>
      <field name="country.countryName" type="strings"/>
      <field name="creationDate" type="tlongs"/>
      <field name="currency" type="string" indexed="true" stored="true"/>
      ***<field name="documents.fileExists" type="booleans"/>
      <field name="documents.fileName" type="strings"/>
      <field name="documents.language" type="strings"/>
      <field name="documents.researchId" type="strings"/>***
      <field name="opinion.opinion" type="strings"/>
      <field name="opinion.opinionId" type="strings"/>
       <field name="employeId" type="string" multiValued="false" indexed="true" stored="true"/>
      <field name="s.sId" type="tlongs"/>
      <field name="s.sName" type="strings"/>
      <field name="type" type="string" indexed="true" stored="true"/>

Thanks all for your comments which helped me to understand Solr more.