实现同步的addAll到Java中的列表

Question

Updated the question.. please check secodn part of question

I need to build up a master list of book ids. I have multiple threaded tasks which brings up a subset of book ids. As soon as each task execution is completed, I need to add them to the super list of book ids. Hence I am planning to pass below aggregator class instance to all of my execution tasks and have them call the updateBookIds() method. To ensure it's thread safe, I have kept the addAll code in synchronized block.

Can any one suggest is this same as Synchronized list? Can I just say Collections.newSynchronizedList and call addAll to that list from all thread tasks? Please clarify.

public class SynchronizedBookIdsAggregator {
    private List<String> bookIds;

    public SynchronizedBookIdsAggregator(){
        bookIds = new ArrayList<String>();
    }

    public void updateBookIds(List<String> ids){
        synchronized (this) {
            bookIds.addAll(ids);
        }
    }

    public List<String> getBookIds() {
        return bookIds;
    }

    public void setBookIds(List<String> bookIds) {
        this.bookIds = bookIds;
    }
}

Thanks, Harish

Second Approach

So after below discussions, I am currently planning to go with below approach. Please let me know if I am doing anything wrong here:-

public class BooksManager{
    private static Logger logger = LoggerFactory.getLogger();

    private List<String> fetchMasterListOfBookIds(){    
        List<String> masterBookIds = Collections.synchronizedList(new ArrayList<String>());
        List<String> libraryCodes = getAllLibraries();

        ExecutorService libraryBookIdsExecutor = Executors.newFixedThreadPool(BookManagerConstants.LIBRARY_BOOK_IDS_EXECUTOR_POOL_SIZE);
        for(String libraryCode : libraryCodes){
            LibraryBookIdsCollectionTask libraryTask = new LibraryBookIdsCollectionTask(libraryCode, masterBookIds);
            libraryBookIdsExecutor.execute(libraryTask);
        }
        libraryBookIdsExecutor.shutdown();

        //Now the fetching of master list is complete.
        //So I will just continue my processing of the master list

    }
}

public class LibraryBookIdsCollectionTask implements Runnable {
    private String libraryCode;
    private List<String> masterBookIds;

    public LibraryBookIdsCollectionTask(String libraryCode,List<String> masterBookIds){
        this.libraryCode = libraryCode;
        this.masterBookIds = masterBookIds;
    }

    public void run(){
        List<String> bookids = new ArrayList<String>();//TODO get this list from iconnect call
        synchronized (masterBookIds) {
            masterBookIds.addAll(bookids);
        }
    }
}

Thanks, Harish

Answer 1

Can I just say Collections.newSynchronizedList and call addAll to that list from all thread tasks?

If you're referring to Collections.synchronizedList , then yes, that would work fine. That will give you a object that implements the List interface where all of the methods from that interface are synchronized, including addAll .

Consider sticking with what you have, though, since it's arguably a cleaner design. If you pass the raw List to your tasks, then they get access to all of the methods on that interface, whereas all they really need to know is that there's an addAll method. Using your SynchronizedBookIdsAggregator keeps your tasks decoupled from design dependence on the List interface, and removes the temptation for them to call something other than addAll .

In cases like this, I tend to look for a Sink interface of some sort, but there never seems to be one around when I need it...

Answer 2

The code you have implemented does not create a synchronization point for someone who accesses the list via getBookIds() , which means they could see inconsistent data. Furthermore, someone who has retrieved the list via getBookIds() must perform external synchronization before accessing the list. Your question also doesn't show how you are actually using the SynchronizedBookIdsAggregator class, which leaves us with not enough information to fully answer your question.

Below would be a safer version of the class:

public class SynchronizedBookIdsAggregator {
    private List<String> bookIds;

    public SynchronizedBookIdsAggregator() {
        bookIds = new ArrayList<String>();
    }

    public void updateBookIds(List<String> ids){
        synchronized (this) {
            bookIds.addAll(ids);
        }
    }

    public List<String> getBookIds() {
        // synchronized here for memory visibility of the bookIds field
        synchronized(this) {
            return bookIds;
        }
    }

    public void setBookIds(List<String> bookIds) {
        // synchronized here for memory visibility of the bookIds field
        synchronized(this) {
            this.bookIds = bookIds;
        }
    }
}

As alluded to earlier, the above code still has a potential problem with some thread accessing the ArrayList after it has been retrieved by getBookIds() . Since the ArrayList itself is not synchronized, accessing it after retrieving it should be synchronized on the chosen guard object:

public class SomeOtherClass {
    public void run() {
        SynchronizedBookIdsAggregator aggregator = getAggregator();
        List<String> bookIds = aggregator.getBookIds();
        // Access to the bookIds list must happen while synchronized on the
        // chosen guard object -- in this case, aggregator
        synchronized(aggregator) {
            <work with the bookIds list>
        }
    }
}

---

I can imagine using Collections.newSynchronizedList as part of the design of this aggregator, but it is not a panacea. Concurrency design really requires an understanding of the underlying concerns, more than "picking the right tool / collection for the job" (although the latter is not unimportant).

Another potential option to look at is CopyOnWriteArrayList .

---

As skaffman alluded to, it might be better to not allow direct access to the bookIds list at all (eg, remove the getter and setter). If you enforce that all access to the list must run through methods written in SynchronizedBookIdsAggregator , then SynchronizedBookIdsAggregator can enforce all concurrency control of the list. As my answer above indicates, allowing consumers of the aggregator to use a "getter" to get the list creates a problem for the user of that list: to write correct code they must have knowledge of the synchronization strategy / guard object, and furthermore they must also use that knowledge to actively synchronize externally and correctly.

---

Regarding your second approach. What you have shown looks technically correct (good!).

But , presumably you are going to read from masterBookIds at some point, too? And you don't show or describe that part of the program! So when you start thinking about when and how you are going to read masterBookIds (ie the return value of fetchMasterListOfBookIds() ), just remember to consider concurrency concerns there too! :)

If you make sure all tasks/worker threads have finished before you start reading masterBookIds , you shouldn't have to do anything special.

But, at least in the code you have shown, you aren't ensuring that.

Note that libraryBookIdsExecutor.shutdown() returns immediately. So if you start using the masterBookIds list immediately after fetchMasterListOfBookIds() returns, you will be reading masterBookIds while your worker threads are actively writing data to it, and this entails some extra considerations.

Maybe this is what you want -- maybe you want to read the collection while it is being written to, to show realtime results or something. But then you must consider synchronizing properly on the collection if you want to iterate over it while it is being written to.

If you would just like to make sure all writes to masterBookIds by worker threads have completed before fetchMasterListOfBookIds() returns, you could use ExecutorService.awaitTermination (in combination with .shutdown() , which you are already calling).

Answer 3

Collections.SynchronizedList (which is the wrapper type you'd get) would synchronize almost every method on either itself or a mutex object you pass to the constructor (or Collections.synchronizedList(...) ). Thus it would basically be the same as your approach.

Answer 4

All the methods called using the wrapper returned by Collections.synchronizedList() will be synchronized. This means that the addAll method of normal List when called by this wrapper will be something like this :-

synchronized public static <T> boolean addAll(Collection<? super T> c, T... elements)

So, every method call for the list (using the reference returned and not the original reference) will be synchronized.

However, there is no synchronization between different method calls. Consider following code snippet :-

 List<String> l = Collections.synchronizedList(new ArrayList<String>);
 l.add("Hello");
 l.add("World");

While multiple threads are accessing the same code, it is quite possible that after Thread A has added "Hello", Thread B will start and again add "Hello" and "World" both to list and then Thread A resumes. So, list would have ["hello", "hello", "world", "world"] instead of ["hello", "world", hello", "world"] as was expected. This is just an example to show that list is not thread-safe between different method calls of the list. If we want the above code to have desired result, then it should be inside synchronized block with lock on list (or this).

However, with your design there is only one method call. SO IT IS SAME AS USING Collections.synchronizedList() .

Moreover, as Mike Clark rightly pointed out, you should also synchronized getBookIds() and setBookIds(). And synchronizing it over List itself would be more clear since it is like locking the list before operating on it and unlocking it after operating. So that nothing in-between can use the List.