如何在Java中发送异步请求并收集响应

Question

I have a list of lets say 2000 user ids and i want to send asynchronous requests to the same server passing the id as a parameter to the WebTarget .

The System will return JSON which is mapped to class UserReport. It includes a List<User> and will be of size 1 (if user was found).

For every result i would like to save the user in a DataBase. Is there a way to make it all Asynchronous? ie As soon as i get a response send an INSERT to the Database.

My code to send a single request:

public UserReport runRequest(String id){
    return this.webTarget.path("some_path")
            .queryParam("id", id)
            .request(MediaType.TEXT_PLAIN_TYPE)
            .get(new GenericType<UserReport >() {});
}

Last question.. Use Observable or Future?

Answer 1

First, List<User> with size 0 or 1 can change to Optional<User> .

You can use flatMap to scheduler async tasks. Here is my code.

public class Q43912265 {

  static Random random = new Random();

  public static void main(String[] args) {
    Observable.range(1, 1000)
        .map(String::valueOf)
        .flatMap(id ->
            Observable.just(id)
                .observeOn(Schedulers.io())
                .map(Q43912265::runRequest))
        .filter(ur -> ur.getUser().isPresent())
        .doOnNext(Q43912265::insert)
        .subscribe();
  }

  @SneakyThrows(InterruptedException.class)
  static UserReport runRequest(String id) {
    System.out.printf("request %s on %s\n", id, Thread.currentThread());
    Thread.sleep(random.nextInt(1000));
    System.out.printf("done %s on %s\n", id, Thread.currentThread());
    return new UserReport(id, Optional.ofNullable(random.nextDouble() > 0.7 ? null : new User(random.nextInt())));
  }

  static void insert(UserReport ur) {
    System.err.printf("insert %s on %s\n", ur, Thread.currentThread());
  }
}

@Value
class UserReport {
  String id;
  Optional<User> user;
}

@Value
class User {
  int id;
}

Note that if you run code above directly, it will terminate before all tasks done. To make it blocking.

And in this case, you can change schedulers with limited thread or io scheduler will create about 1000 thread.

Answer 2

I finally solved it using ExecutorService and Future

I post the answer:

public List<User> getAllUsers(List<String> ids) {

    List<Future<UserReport>> futures = new ArrayList<>();
    ExecutorService executor = Executors.newFixedThreadPool(10);

    int counterU = 0;
    for (String id : ids) {
        Callable<UserReport> task = () -> {
            return runRequest(id);
        };
        futures.add(executor.submit(task));
        LOGGER.info("Added Task {} for UserId {}.", counterH++, id);
    }

    List<User> toReturn = new ArrayList<>();

    for (Future<UserReport> future : futures) {
        try {
            UserReport report = future.get();

            if (report.getUsers() != null) {
                User temp = report.getUsers().get(0);
                LOGGER.info("Got details for User {}.", temp.getId());
                toReturn.add(temp);
                insertUserToDB(temp);
            }

        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (ExecutionException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
    executor.shutdown();
    return toReturn;
}