How to handle pagination with Retrofit 2 and RxJava

Question

I know how to handle Retrofit responses but I have a problem with handling pagination from a REST API with rx java.

Background

The rest api that I'm using gives me the following response with the link to the next page in the headers:

HTTP 200 OK
Allow: GET, HEAD, OPTIONS
Content-Type: application/json
Link: <http://some-endpoint.com/pictures?page=2>; rel="next"
Vary: Accept
X-Total-Count: 80

[
    {
        "id": 3,
        "url": "",
        "picture": {
            "id": 6,
            "url": "",
            "name": "Chrysler"
        },
        "location": {
            "type": "Point",
            "coordinates": [
                10.108956,
                41.389576000000005
            ]
        }
    },
    {
        "id": 4,
        "url": "",
        "picture": {
            "id": 26,
            "url": "",
            "name": "Douglas Aircraft"
        },
        "location": {
            "type": "Point",
            "coordinates": [
                10.1977759999999997,
                41.426645
            ]
        }
    },
...
}

Right know I managed to get the pagination request to page 2 with the following code but I don't know how to make those pagination requests until there is no "next" header link on the response so I can be able to save all data from all requests to the database.

private Observable<List<PictureEntity>> getPictures(final LocationEntity locationEntity) {
    if (getDataFromServer()) {
        return mApiService.getPictures()
                .concatMap(new Func1<Response<List<Picture>>, Observable<Response<List<Picture>>>>() {
                    @Override
                    public Observable<Response<List<Picture>>> call(Response<List<Picture>> listResponse) {
                        String link = NetworkUtil.getNextLinkPaginate(listResponse.headers().get("Link"));
                        Timber.i(link);
                        return Observable.just(listResponse)
                                .concatWith(mApiService.getPicturesPaginate(link))
                                .reduce(new Func2<Response<List<Picture>>, Response<List<Picture>>,
                                        Response<List<Picture>>>() {
                                    @Override
                                    public Response<List<Picture>> call(Response<List<Picture>> listResponse,
                                                                            Response<List<Picture>> listResponse2) {
                                        listResponse.body().addAll(listResponse2.body());
                                        return listResponse;
                                    }
                                });
                    }
                })
                .concatMap(new Func1<Response<List<Picture>>, Observable<List<PictureEntity>>>() {
                    @Override
                    public Observable<List<PictureEntity>> call(Response<List<Picture>> listResponse) {
                        List<PictureEntity> pictureEntities =
                                mPictureDataMapper.transformToEntity(listResponse.body());
                        mDatabaseHelper.setPictureEntity(pictureEntities).subscribe();
                        return mDatabaseHelper.getNearbyPicturesEntity(locationEntity);
                    }
                });
    } else {
        return mDatabaseHelper.getNearbyPicturesEntity(locationEntity);
    }
}

Any suggestions on how to handle this kind of requests? Maybe .takeUntil operator?

Answer 1

You could use concatMap with a bit of recursion.

Observable<List<PointOfSaleEntity>> getPointsOfSalePaginateUntilEnd(String link) {
  if (link.equals("")) {
      return Observable.empty();
  } else {      
      return mApiService.getPointsOfSalesPaginate(link)                     
                        .concatMap(listResponse -> Observable.concat(Observable.just(listResponse),mApiService.getPointsOfSalesPaginate(NetworkUtil.getNextLinkPaginate(listResponse.headers().get("Link")))));
  }
}

Observable<List<PointOfSaleEntity>> getPointsOfSalePaginateUntilEnd() {   
       return mApiService.mApiService.getPointsOfSales()                        
                         .concatMap(listResponse -> Observable.concat(Observable.just(listResponse),mApiService.getPointsOfSalesPaginate(NetworkUtil.getNextLinkPaginate(listResponse.headers().get("Link")))));
}