简体   繁体   中英

Returning data from multiple threads Vs accumulating responses

I am working on a simple project that is going to send multiple HTTP requests to retrieve some data, parse the response from each URL, and return response that contains the original URL and some information about the data(The reason for threads usage is obviously the multiple HTTP requests).
I am wandering if there is best practice for this scenario, here are the options that pops into my mind :
1. Have each thread send an HTTP request. parse the data to get the required information and return the information itself (by a Future<SomeDataType> , or a simple DataType getInformation() call to be done after the thread is complete), then create the URL-SomeDataType pair in the original thread
2. Having each thread take an additional argument of a synchronized list/map, which the thread will add the URL-Information pair to (the same instance of the list/map will be shared across all threads).
3. Less likely option - having each thread just pull the information, and return in in either way mentioned in 1/2, than parsing all the information in the main thread (which will reduce performance but will require almost 0 synchronisation handling, which is nice)

Is there a best practice for a similar scenario?
Thanks!

In my opinion, Option 1 is the cleanest and aligns with the best practice. Preferred way to implement it would be to use the executor framework (thread-pools and Callables). Reasons for the choice -

  1. Separation of concerns - each thread returns the results of its' work independently. After that, it's the main thread's job to take that result and process it further the way it likes (eg put it in a map OR merge it into something else). In the future, if you found a better/cleaner way of aggregating the results - that change would most-likely not impact what the worker threads themselves do or return.

  2. Option 2 would involve unnecessary synchronization (although you could use ConcurrentHashMap to make it minimal). Bigger problem - it mixes the concerns among the main thread and the worker threads. The worker threads now "know" a bit about what to with the result (their concern should only be - getting the results)

  3. Option 3, as you said, would degrade performance. If the information fetched by each thread is independent from each other, it makes sense to let each thread parse that info and then return it.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM