Using RxJava + Retrofit to make API requests for each item in a list

Question

I'm trying to create an observable by chaining together multiple retrofit api calls. The steps are:

1. Get a list of json objects using api call
2. For each object in the list, make another api call to get further details about the item
3. Write data taken from this new detailed object to a file on the disk (this occurs for each item in the list)
4. Finally return an observable of a separate object that requires that a file was created for each of the previous objects

This is what I have so far:

public static Observable<DownloadedFiles> downloadFiles() {
    DownloadedFiles downloadedFiles = new DownloadedFiles();
    Observable.create(subscriber -> {
        return getRestService().getObjectList()
        .flatMapIterable(objects -> objects)
        .flatMap(objectLimited -> getRestService().getObject(objectLimited.getPath()))
        .doOnNext(objectFull -> {

            try {
                File file = new File();
                // Extract data from objectFull and write new file to disk
                // ...
                } catch (IOException e) {
                subscriber.onError(e);
            }

            downloadedFiles.putFile(file);
        })
        .toList()
        .map(objects -> downloadedFiles)
        .finallyDo(() -> {
            subscriber.onNext(downloadedFiles);
            subscriber.onCompleted();
        });
    });
}

@GET("/api/...")
Observable<List<Object>> getObjectList();

@GET("/api/.../{path}")
Observable<Object> getObject(@Path("path") String path);

Could someone please confirm that I have used the correct operators. Thanks .

Answer 1

Edit: Removed the Observable.create, retrofit is already making an observable for you, you merely need to transform it.

Edit 2: you also shouldn't need to do anything with subscriber.onError, if an error gets thrown it will call subscriber.onError on its own.

Pretty good, Not sure why you went with flatmap observable. I would instead do flatmap to Observable::from, also collect is worth adding. Basically I'm going to map 1 thing to many, then do some action, collect the many back into one and then subscribe to that one once I collected all items emitted.

public static Observable<DownloadedFiles> downloadFiles() {        
        return getRestService().getObjectList()
        .flatMap(Observable::from)
        .flatMap(objectLimited -> getRestService().getObject(objectLimited.getPath()))
        .doOnNext(objectFull -> {
            try {
                File file = new File();
                // Extract data from objectFull and write new file to disk
                // ...
            } catch (IOException e) {
                new IOException(e);
            }})
      .collect(() -> new DownloadFiles<>(), (files, object) -> {  files.add(object});

Answer 2

I think some like this should work for you. Instead of pulling in a Subject to emit it the DownloadedFile(s) to, you can just use the Observable from the REST service to map to each of the DownloadedFiles:

public static Observable<DownloadedFile> downloadFiles() {
    final Observable<Observable<FullObject>> observable =  getRestService().getObjectList()
            .flatMapIterable(objects -> objects)
            .map(objectLimited -> getRestService().getObject(objectLimited.getPath()));

    return  Observable.mergeDelayError(observable)
            .map(fullObject -> {
                try {
                    File file = new File("path");
                    // Extract data from objectFull and write new file to disk
                    // ...

                    return new DownloadedFile();
                } catch (IOException e) {
                    throw OnErrorThrowable.from(OnErrorThrowable.addValueAsLastCause(e, fullObject));
                }
            });
}

You may want to consider using mergeDelayError(map()) instead of flatMap if you want to emit successfully saved files before propagating any errors.