设计具有低线程争用的多线程聊天服务器

Question

I was thinking how I would make a multi-threaded chat server with C++ in a some way that minimizes thread contention.

In my initial design I have an std::vector of sockets in the server. When a client connects to the server, the socket is added to this vector of sockets.

There is also an std::unordered_map<string, Socket*> that allows to look up the corresponding socket for a user name. When the client logs in with it's user and password we add an entry to the hash map. When the user logs out we delete the corresponding entry in the hash map.

The client will send messages addressed to a user name. When they get to the server we use the hash map to look up the socket, and send the message though that socket.

Since the server is multi-threaded, and the mentioned data structures can be read/written from different threads, we now need to guard them with some thread synchronization mechanism such as a mutex. But I think doing so would kill performance because of thread contention. Basically, all threads need to access these data structures in order to send messages, but only one of them can use them at the same time. I think with this approach performance wouldn't be much better than doing it with a single thread.

How can I improve my design for better performance?

Answer 1

I think with this approach performance wouldn't be much better than doing it with a single thread.

Not necessarily. Since your map is a map of pointers , and not a map of objects Accessing the table is not the same thing as acessing the socket, and protecting the former does not mean that the later needs to be protected as well, even if it lives within the data structure.

However, you will need to make sure that the lifetime of the object is safely handled. This is one of the cases where std::shared_ptr<> is your friend, as it guarantees thread safe ownership safety.

For example:

std::mutex table_mtx;
std::unordered_map<string, std::shared_ptr<Socket>> sockets;

void send(const std::string& msg, const std::string& dst_name) {
  std::shared_ptr<Socket> dst;
  {
    std::lock_guard<std::mutex> lock(table_mtx);

    // Increments the ref-count on the socket, so even if it's removed 
    // from the map, it won't be deleted until we are done with it. 
    dst = sockets.at(dst_name);
  }

  if(dst) {
    dst->send(msg);
  }
}

Obviously, Socket also needs to be have an internal mutex to handle contention when using the same socket concurently. However, if user1 sends a message to user2 while user3 sends a message to user4, the contention would be limited to the lookup within the map, while the rest of the operation would be concurent.

Answer 2

Since chat has high temporal correlation (because conversation) the logical answer is to cache results. You need a weak ptr for invalidation.

Answer 3

The simple solution is to create a reference-counted message class and use message queues. If Alice wants to send a message to Bob and Charlie, you create an instance of the reference-counted message class, and then call a "queue message" function to queue instances of that same message to both Bob and Charlie.

The "queue message" functions works as follows:

1. Acquire the client map lock.
2. Find the client.
3. Lock the client send queue lock.
4. Release the client map lock.
5. Add the message to the client's send queue.
6. If the send queue was empty, call an async send function.
7. Release the client send queue lock.

The majority of the work your server does will be outside this "queue message" function entirely. All of the sending, parsing, and receiving can take place without holding any locks at all. When you receive a message, you can follow the same logic:

1. Receive data.
2. Parse it into a message.
3. Acquire the client's receive queue lock.
4. Put the message on the client's receive queue.
5. If the receive queue was empty, dispatch the client's message processing engine.
6. Release the client's receive queue lock.

The receive queue dispatch logic:

1. Acquire the client's receive queue lock.
2. If the queue is empty, release the lock and stop.
3. Pull a message off the client's receive queue.
4. Release the client's receive queue lock so newly-received messages can be queued.
5. Process the received message.
6. Go to step 1.

By the way, I was the primary developer of WebMaster's ConferenceRoom software. So I've done this. Handling ten thousand clients this way on hardware that's more than a decade old was no problem. Today, I would use boost to do much of the work for me.

Answer 4

First and simple solution:
If you have enough resources on the server or not many clients, I suggest to avoid most of the multi-thread complexity here and place all sending or receiving functionality to a single thread (one for send and another for receive operation). So, threads have their working socket and only locks of send and receive clients' queues remain. These locks can be handled by Producer/consumer pattern.

More advanced but also more complex solution: You must use more optimized structures. Using "unordered_map" object makes your socket search mechanism very inefficient. Also, you should not use exclusive locks everywhere that locks are needed, consider using nonexclusive locks anywhere possible as well.
Anyway, it is better to harness existing thread-safe and lock-free libraries. You can find many of them online. I search one on Google for you:
https://github.com/khizmax/libcds