Using Cloudant Client Search API Doesn't Return All Results Expected

Question

I am pretty new to Cloudant but have developed in SQL on DB2 for some time. I am running into an issue where I *think I am using the Lucene query engine and Cloudant indexes to return results from my query, but only two out of three expected fields are returning data. The field that isn't returning data is an array. Our application is running Java and executed using IBM's Bluemix and WebSphere Liberty Profile. I have packaged the cloudant-client-2.8.0.jar and cloudant-HTTP-2.8.0.jar files to access the cloudant database. We have many queries that are working so the connection itself is fine.

Here is my code that accesses the Cloudant API:

....
Search searchSaaSData = ServiceManagerSingleton.getInstance().getSaaSCapabilitiesDatabase().search("versioning-ddoc/versioning-indx").includeDocs(true);

SearchResult<DocTypeInfo> result = searchSaaSData.querySearchResult(this.getJsonSelector(deliverableId), DocTypeInfo.class);
....

Here is the this.getJsonSelector code:

private String getVersioningSearch(String deliverableId) {
    String query = "";
    if (deliverableId != null && !deliverableId.isEmpty()) {
        // Search based on deliverableId
        query += "(";
        query += "deliverableId:\"" + deliverableId + "\"" + ")";
    }

    logger.log(Level.INFO, "Search query is: " + query);
    // The query is simple, will look like this: 
    // deliverableId:"0B439290AB5011E6BE74C84817AAB206")

    return query;
}

As you can see from the java code above I am using the java object DocTypeInfo.class to hold the data returned from the search. Here is the class:

public class DocTypeInfo {
    private final String docType;
    private final String version;
    private String isPublishedForReports;

    public DocTypeInfo(String docType, String version) {
        this.docType = docType;
        this.version = version;
    }

    /**
     * @return the docType
     */
    private String getDocType() {
        return docType;
    }

    /**
     * @return the version
     */
    private String getVersion() {
        return version;
    }

    /**
     * @return the isPublishedForReports
     */
    public String getIsPublishedForReports() {
        return isPublishedForReports;
    }

    /**
     * @param isPublishedForReports the isPublishedForReports to set
     */
    public void setIsPublishedForReports(String isPublishedForReports) {
        this.isPublishedForReports = isPublishedForReports;
    }       

}

I have setup a design doc and index using the Cloudant dashboard as follows:

{
"_id": "_design/versioning-ddoc",
"_rev": "22-0e0c0ccfc2b5fe7245352da7e5b1ebd3",
"views": {},
"language": "javascript",
"indexes": {
  "versioning-indx": {
  "analyzer": {
    "name": "perfield",
    "default": "standard",
    "fields": {
      "deliverableId": "whitespace",
      "docType": "standard",
      "version": "standard",
      "isPublishedForReports": "keyword"
    }
  },
  "index": "function (doc) {
               index(\"deliverableId\", doc.deliverableId, {\"store\":true, \"boost\":1.0});
   index(\"docType\", doc.docType, {\"store\":true, \"boost\":1.0});
   index(\"version\", doc.version, {\"store\":true, \"boost\":1.0});
   if (Array.isArray(doc.publishSettings)) {
     for (var i in doc.publishSettings) {
       if (doc.publishSettings[i].options) {
         for (var j in doc.publishSettings[i].options) {
           index(\"isPublishedForReports\", doc.publishSettings[i].options[j].optionId, {\"store\":true, \"boost\":1.0});
           }
         }
       }
     }
   }"
  }
  }
 }

When I execute the search the docType and version fields are populated, however the isPublishedForReports field is ALWAYS null or doesn't return. When I run a query in the Cloudant dashboard against the index I can see the isPublishedForReports value is returned, I don't know why it's not populated in the object? Maybe I am misunderstanding how these get built?

Here a screenshot where I query the DB and I can see the results I want:

Please help! -Doug

Answer 1

I think you are accessing the docs property from each row in result.rows instead of the fields property. When you run your search with .includeDocs(true) you will see a result similar to the following:

{
  "total_rows":3,
  "bookmark":"g1AAA",
  "rows":[
    {
      "id":"263a81ea76528dead3a4185df3676f62",
      "order":[
        1.0,
        0
      ],
      "fields":{
        "docType":"xxx",
        "deliverableId":"yyy",
        "isPublishedForReports":"pu_1_2"
      },
      "doc":{
        "_id":"263a81ea76528dead3a4185df3676f62",
        "_rev":"3-116dd3831c182fb13c12b05a8b0996e4",
        "docType":"xxx",
        "deliverableId":"yyy",
        "publishSettings":[...]
      }
    }
    ...
  ]
} 

Notice how you can see the fields you defined in your search index in the fields property and the full document in the doc property. The full document does not include isPublishedForReports .

To get the isPublishedForReports value you need to access the fields property:

for (SearchResult<DocTypeInfo>.SearchResultRow row : result.getRows()) {
   ...
   row.getFields().getIsPublishedForReports()
   ...
}

Also, if you don't need the whole doc you can set .includeDocs(false) and only access the fields property.