使用JSON与Java查询elasticsearch

Question

I'm using elasticsearch 1.3.1 and i want to query it with java SPRING & elastic java API.

I use Angularjs for front-end and i can query elasticsearch very well.

What i want to do is :

1) create the json query for elasticsearch with angularjs : ok

    {
    "bool" : {
        "must" : [{"query_string" : {   "query" : "***"}}],
        "must_not" : [
            {"term" : {"O_LIFESTAGE" : "lymphe" }}, 
            {"term" : {"O_SEX" : "m"}}, 
            {"term" : {"L_CONTINENT" : "europe" }}, 
            {"term" : {"I_INSTITUTIONCODE" : "herbierparis"}}, 
            {"term" : {"I_INSTITUTIONCODE" : "herbiercaen"}}, 
            {"term" : {"I_INSTITUTIONCODE" : "herbiertoulouse"}}
        ]
    }
}

2) to call a java SPRING rest web service and send it the json query created in 1) : ok

3) the web service query elasticsearch with the query created in 1) and save the result ( list of ids) : i failed, i get an error ( nested exception is org.elasticsearch.indices.IndexMissingException: [donnees] missing ) with the follow code :

Node node = nodeBuilder().clusterName("elasticsearch noeud 1").node();
Client client = node.client();
String myquery = "{\"bool\": {\"must\": [{\"query_string\": {\"query\": \"***\"}}],\"must_not\": [{\"term\": {\"O_LIFESTAGE\": \"lymphe\"}},{\"term\": {\"O_SEX\": \"m\"}},{\"term\": {\"L_CONTINENT\": \"europe\"}},{\"term\": {\"I_INSTITUTIONCODE\": \"herbierparis\"}},{\"term\": {\"I_INSTITUTIONCODE\": \"herbiercaen\"}},{\"term\": {\"I_INSTITUTIONCODE\": \"herbiertoulouse\"}}]}}";
//WrapperQueryBuilder querybuilder= new WrapperQueryBuilder(query) ;

//try to get it work with an easy query before trying with "myquery"
SearchResponse searchResponse = client.prepareSearch("donnees").setTypes("specimens").setQuery("{\"term\": {\"L_CONTINENT\": \"europe\"}}").execute().actionGet();

SearchHit[] results = searchResponse.getHits().getHits();
System.out.println("Current results: " + results.length);
for (SearchHit hit : results) {
    System.out.println("------------------------------");
    Map<String,Object> result = hit.getSource();   
    System.out.println(result);
}
node.close();

---

---

this is how i create my elasticsearch index :

 curl -XPOST 'localhost:9200/donnees'

this is the index mapping :

curl -XPUT 'localhost:9200/donnees/specimens/_mapping' -d '{
"specimens" : {
    "_all" : {"enabled" : true},
    "_index" : {"enabled" : true},
    "_id" : {"index": "not_analyzed", "store" : false},
    "properties" : { ... }}}'

the data import :

curl -XPUT 'localhost:9200/_river/donnees_s/_meta' -d '{
 "type" : "jdbc",
 "jdbc" : {
    "index" : "donnees",
    "type" : "specimens",
    "url" : "jdbc:oracle:thin:@localhost:1523:recolnat",
     "user" : "user",
     "password" : "pass",
     "sql" : "select * from all_specimens_data"
 }}'

Example of query i try to do in java, working well in curl & JS :

curl -XGET 'http://localhost:9200/donnees/specimens/_search?size=100000000' -d '
{
"fields" : ["O_OCCURRENCEID"],
"query" :     {
    "bool" : {
        "must" : [{"query_string" : {   "query" : "***"}}],
        "must_not" : [
            {"term" : {"O_LIFESTAGE" : "lymphe" }}, 
            {"term" : {"O_SEX" : "m"}}, 
            {"term" : {"L_CONTINENT" : "europe" }}, 
            {"term" : {"I_INSTITUTIONCODE" : "herbierparis"}}, 
            {"term" : {"I_INSTITUTIONCODE" : "herbiercaen"}}, 
            {"term" : {"I_INSTITUTIONCODE" : "herbiertoulouse"}}
        ]
    }
}}'

---

I can't find what i should put for the "indice" who should be my index (donnees), i even try "donnee_s". Any advice are very very welcomed. And also if someone know who to query directly the full "myquery"

Thank you for your reading and help

Answer 1

The error message is pretty clear : your index doesn't exist.

Are you sure about your cluster name? In your java code, it is set to elasticsearch noeud 1 , which is more like a node name.

You are maybe querying the wrong cluster : try to retrieve the cluster name with a curl on _cluster/state to compare.

Answer 2

working code to query elasticsearch with a jsonquery made by angularjs ( the jsonquery work in java and JS elastic api) :

import static org.elasticsearch.node.NodeBuilder.*;
import org.elasticsearch.action.search.SearchRequestBuilder;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.client.Client;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.node.Node;
import org.elasticsearch.search.SearchHit; 
...

Settings settings = ImmutableSettings.settingsBuilder()
                .put("http.enabled", "false")
                .put("transport.tcp.port", "9300-9400")
                .put("discovery.zen.ping.multicast.enabled", "false")
                .put("discovery.zen.ping.unicast.hosts", "localhost").build();

Node node = nodeBuilder().client(true).settings(settings)
        .clusterName("elasticsearch").node();
Client client = node.client();

//jsonquery is made by angularjs like : "{\"bool\" : {\"must\" : [{\"query_string\" : {\"query\" : \"***\"}}],\"must_not\" : [{\"term\" : {\"O_SEX\" : \"m\"}}, {\"term\" : {\"L_CONTINENT\" : \"amerique\"}}, {\"term\" : {\"I_INSTITUTIONCODE\" : \"herbiertoulouse\"}}]}}";

SearchRequestBuilder searchRequestBuilder = client.prepareSearch("donnees").setSearchType(SearchType.DFS_QUERY_THEN_FETCH).addField("O_OCCURRENCEID").setQuery(jsonquery);

if( limit != null ){    searchRequestBuilder.setSize(limit); }else{ searchRequestBuilder.setSize(250000000); }
if( offset !=null ){    searchRequestBuilder.setFrom(offset); }else{ searchRequestBuilder.setFrom(0); }

SearchResponse searchResponse = searchRequestBuilder.execute().actionGet();

SearchHit[] results = searchResponse.getHits().getHits();
System.out.println("Current results: " + results.length);

int length = results.length;
for (int rnum = 1; rnum<=length;rnum++){
    SearchHit hit = results[rnum-1];
    System.out.println( hit.getFields().get("O_OCCURRENCEID").getValue()  );
}        
node.close();