Java中的同步多线程（Apache HTTPClient）

Question

I am wondering how I would go about doing this. Say I load a list of 1,000 words and for each word a thread is created and say it does a google search on each word. The problem here is obvious. I can't have 1k threads, can I. Keep in mind I am extremely new to threads and synchronization. So basically I am wondering how I would go about using less threads. I assume I have to set thread amount to a fixed number and synchronize the threads. Was wondering how to do this with Apache HttpClient using GetThread and then run it. In run I'm getting the data from webpage and turning it into a String and then checking if it contains a certain word.

Answer 1

Surely you can have as many threads as you want. But in general it is not recommended to use more threads than there are processing cores on your computer. And don't forget that creating 1000 internet sessions at once affects your networking. A size of one single google page is nearly 0.3 megabytes. Are you really going to download 300 megabytes of data at once?

By the way,

There is a funny thing about concurrency. Some people say: "synchronization is like concurrency". It is not true. Synchronization is the opposite of concurrency. Concurrency is when lots of things happen in parallel. Synchronization is when I am blocking you. (Joshua Bloch)

Answer 2

Maybe you can look at this problem this way.

You have 1000 words and for each word you are going to carry out a search. In other words there are 1000 tasks to be executed and they are not related to each other, so there is no need for synchronization in the case of this problem as per the following definition from Wiki.

"In computer science, synchronization refers to one of two distinct but related concepts: synchronization of processes, and synchronization of data. Process synchronization refers to the idea that multiple processes are to join up or handshake at a certain point, in order to reach an agreement or commit to a certain sequence of action. Data Synchronization refers to the idea of keeping multiple copies of a dataset in coherence with one another, or to maintain data integrity"

So in this problem you do not have to synchronize the 1000 processes which execute the word searches since they can run independently and dont need to join forces. So it is not a Process synchronization.

It is not a Data synchronization either since the data of each search is independent of the other 999 searches.

Hence when Joshua says Synchronization is when I am blocking you, there is no need of blocking in this case.

Yes all tasks can concurrently get executed in different threads. Of course your system may not have the resources to run 1000 threads concurrently ( read same time ). So you need concepts like pools where a pool has a certain no of threads...say if it has 10 threads...then those 10 will start 10 independent searches on 10 words from your list. If any of them is done with its task then it will take up the next word search task available and the process goes on....