Bulk read Couchbase documents

Question

I want to asynchronously read a number of documents from a Couchbase bucket. This is my code:

JsonDocument student = bucketStudent.get(studentID);

The problem is for a large data file with a lot of studentIDs, it would take a long time to get all documents for these studentIDs because the get() method is called for each studentID. Is it possible to have a list of studentIDs as input and return an output of a list of students instead of getting a single document for each studentID?

Answer 1

AFAIK couchbase SDK does not have a native function for a bulk get operation.

The node.js SDK has a getMulti method, but it's basically an iteration over an array and then get() is fired for each element.

I've found in my applications that the key-value approach is still faster than the SELECT * on a primary index but the N1QL query is remarkably close (on couchbase 5.x).

Just a quick tip: if you have A LOT of ids to fetch and you decide to go with the N1QL queries, try to split that list in smaller chunks. It's speeds up the queries and you can actually manage your errors better and avoid getting some nasty timeouts.

Answer 2

If you are running a query node, you can use N1QL for this. Your query would look like this:

SELECT * FROM myBucket USE KEYS ["key1", "key2", "key3"]

In practice you would probably pass in the array of strings as a parameter, like this:

SELECT * FROM myBucket USE KEYS ?

You will need a primary index for your bucket, or queries like this won't work.

Answer 3

Retrieving multiple documents using the document IDs is not supported by default in the Couchbase Java SDK. To achieve that you'll need to use a N1QL query as below

SELECT S.* FROM Student S USE KEYS ["StudentID1", "StudentID2", "StudentID3"]

which would return an array of Documents with the given IDs. Construct the query with com.couchbase.client.java.query.N1qlQuery , and use either of below to execute

If you're using Spring's CouchbaseTemplate , you can use the below

List<T> findByN1QL(com.couchbase.client.java.query.N1qlQuery n1ql,
                          Class<T> entityClass)

If you're using Couchbase's Java SDK , you can use the below

N1qlQueryResult query(N1qlQuery query)

Note that you'll need an index on your bucket to run N1QL queries.