网络服务器应用程序中的内存碎片和垃圾回收

Question

Actually i developed an tcp based network server that does a lot of read an write and uses less than zero cpu for its business logic, it acts as bridge between two endpoint. The network server is developed for .NET 4.5, it uses:

• IOCP (Socket.xxxAsync)
• a Buffer Pool of preallocated SocketAsyncEventArgs (used for SendAsync and ReadAsync)
• a Buffer Pool of preallocated byte[] (used only for read)
• System.Collections.Concurrent
• Less than zero locks (well, actually there is one :))
• some other stuff

My concerns are related to the garbage collection, infact, as far i understand, while i avoid the memory fragmentation preallocating all the buffers i need, the garbage collector check if them must be collected or not because they aren't allocated inside the large heap.

Would be better, instead to allocate 10.000 byte[8192] would be better allocate a big byte[81920000] and use ArraySegment to use slices?

Thank you.

UPDATE

I switched to server mode garbage collection and it seems that my system works better than before (actually i'm able to handle up to 4GiB of traffic with 5000 clients on the same machine, this means 10000 sockets). I'll start to test the architecture on more machine in short.

Answer 1

You need to be careful with how you break down data for sending and receiving. When receiving data try to keep any buffers below the minimum for the LOH, ie <80Kb. Same for sending.

An alternative is to create a pool of buffers of a fixed size to use and recycle as necessary. This avoid the constant creation and destruction of allocation avoiding any memory fragmentation issues.

The final choice, as with most problems, depends on what's optimum for your final solution. It can be easier to avoid fragmentation by just using small buffers but this can also have an associated performance cost.

Answer 2

I had this "memory leak"-like problem when I was developing a high performance TCP server based on SocketAsyncEventArgs .

The problem is when you use a buffer ( byte[] ) it gets pinned . So actually GC can not do anything about it.

Personally I wrote this class (not exactly this):

class Buffer
{
    const int BUFFER_SIZE = 8 * 1024;

    public Buffer()
    {
        InUse = false;
        Bytes = new byte[BUFFER_SIZE];
    }

    public bool InUse { get; set; }
    public byte[] Bytes { get; private set; }
}

And another class named BufferPool which has List<Buffer> as a pool of buffers (the logic for detecting dead connections and free their pool and etc is very complicated in my case so I skip the internals).

That way you can reuse the allocated byte array as a buffer for a new action.

I have implemented it this way because I could not restrict the max number of connections. If you can restrict the number of connections to a specific max then this codeproject article can help.

Note: I forgot to say there is no way to reclaim the allocated memory in this case.

Answer 3

Client-server application in simple case means N clients and 1 server. Each of the client requests shall be self-contained and independent from other clients' requests. This allows you to use N thread to support N concurrent users.

Now, it is essential that threads don't use shared resources. They could, but it comes as a performance hit. Even if you use optimistic (lockless) concurrency model, this doesn't mean threads won't compete for the shared resource.

Now if you have a separate buffer per thread, each thread uses it's own memory and no competition exist between different concurrent threads. If you have many users, this will increase memory fragmentation and CPU time spent by GC.

If you use a large, but shared buffer, you will increase time spent on trying to get access to the shared buffer for different threads. And also you will decrease time spent by GC doing collection stuff.

---

Personally, I would just use 1 small buffer per thread. This has bonuses:

• Simple model
• Less code to write, no synchronization needed
• Your CPU level is small so no need to do performance optimization
• This model is much easier to scale horizontally: add another server and a load balancer. It's working out of the box.