SharpDX 内存碎片

Question

I am working on a .NET 3.5 application which uses SharpDX to render tiled 2D images.

Textures (Texture2D) are loaded into a cache on-demand, and are created in the managed pool.

Textures are disposed of when no longer required, and I have verified that Dispose() is called correctly. SharpDX object tracking indicates that there are no textures being finalized.

The issue is that large amounts of unmanaged heap memory used by the textures continues to be reserved after disposal. This memory is reused when loading a new texture, so memory is not being leaked.

However, another part of the application also requires significant chunks of memory to process new images. Because these heaps are still present, even though the textures have been disposed, there is not enough contiguous memory to load another image (can be hundreds of MB).

If I allocate unmanaged meory using AllocHGlobal , the resulting heap memory completely vanishes again after calling FreeHGlobal .

VMMap shows the unmanaged heap (red) after heavy usage of the application.

We can see here that unmanaged heap accounts for ~380MB, even though only ~20MB is actually committed at this point.

Long term, the application is being ported to 64-bit. However, this is not trivial due to unmanaged dependencies. Also, not all users are on 64-bit machines.

EDIT: I've put together a demonstration of the issue - create a WinForms application and install SharpDX 2.6.3 via Nuget.

Form1.cs:

using System.Collections.Generic;
using System.Diagnostics;
using System.Windows.Forms;
using SharpDX.Direct3D9;

namespace SharpDXRepro {
    public partial class Form1 : Form {
        private readonly SharpDXRenderer renderer;
        private readonly List<Texture> textures = new List<Texture>();

        public Form1() {
            InitializeComponent();

            renderer = new SharpDXRenderer(this);

            Debugger.Break(); // Check VMMap here

            LoadTextures();

            Debugger.Break(); // Check VMMap here

            DisposeAllTextures();

            Debugger.Break(); // Check VMMap here

            renderer.Dispose();

            Debugger.Break(); // Check VMMap here
        }

        private void LoadTextures() {
            for (int i = 0; i < 1000; i++) {
                textures.Add(renderer.LoadTextureFromFile(@"D:\Image256x256.jpg"));
            }
        }

        private void DisposeAllTextures() {
            foreach (var texture in textures.ToArray()) {
                texture.Dispose();
                textures.Remove(texture);
            }
        }
    }
}

SharpDXRenderer.cs:

using System;
using System.Linq;
using System.Windows.Forms;
using SharpDX.Direct3D9;

namespace SharpDXRepro {
    public class SharpDXRenderer : IDisposable {
        private readonly Control parentControl;

        private Direct3D direct3d;
        private Device device;
        private DeviceType deviceType = DeviceType.Hardware;
        private PresentParameters presentParameters;
        private CreateFlags createFlags = CreateFlags.HardwareVertexProcessing | CreateFlags.Multithreaded;

        public SharpDXRenderer(Control parentControl) {
            this.parentControl = parentControl;

            InitialiseDevice();
        }

        public void InitialiseDevice() {
            direct3d = new Direct3D();
            AdapterInformation defaultAdapter = direct3d.Adapters.First();

            presentParameters = new PresentParameters {
                Windowed = true,
                EnableAutoDepthStencil = true,
                AutoDepthStencilFormat = Format.D16,
                SwapEffect = SwapEffect.Discard,
                PresentationInterval = PresentInterval.One,
                BackBufferWidth = parentControl.ClientSize.Width,
                BackBufferHeight = parentControl.ClientSize.Height,
                BackBufferCount = 1,
                BackBufferFormat = defaultAdapter.CurrentDisplayMode.Format,
            };

            device = new Device(direct3d, direct3d.Adapters[0].Adapter, deviceType,
                parentControl.Handle, createFlags, presentParameters);
        }

        public Texture LoadTextureFromFile(string filename) {
            using (var stream = new FileStream(filename, FileMode.Open, FileAccess.Read)) {
                return Texture.FromStream(device, stream, 0, 0, 1, Usage.None, Format.Unknown, Pool.Managed, Filter.Point, Filter.None, 0);
            }
        }

        public void Dispose() {
            if (device != null) {
                device.Dispose();
                device = null;
            }

            if (direct3d != null) {
                direct3d.Dispose();
                direct3d = null;
            }
        }
    }
}

My question therefore is - (how) can I reclaim the memory consumed by these unmanaged heaps after the textures have been disposed?

Answer 1

It seems the memory that's giving the issues is allocated by the nVidia driver. As far as I can tell, all the deallocation methods are properly called, so this might be a bug in the drivers. Looking around the internet shows some issues that seem related to this, though it's nothing serious enough to be worth referencing. I can't test this on an ATi card (I haven't seen one in like ten years :D).

So it seems like your options are:

• Make sure your textures are big enough to never be allocated on the "shared" heaps. This allows the memory leak to proceed much slower - although it's still unreleased memory, it's not going to cause memory fragmentation anywhere near as serious as you're experiencing. You're talking about drawing tiles - this has historically been done with tilesets, which give you a lot better handling (though they also have drawbacks). In my tests, simply avoiding tiny textures all but eliminated the problem - it's hard to tell if it's just hidden or completely gone (both are quite possible).
• Handle your processing in a separate process. Your main application would launch the other process whenever needed, and the memory will be properly reclaimed when the helper process exits. Of course, this only makes sense if you're writing some processing application - if you're making something that actually displays the textures, this isn't going to help (or at least it's going to be really tricky to setup).
• Do not dispose of the textures. Managed texture pool handles paging the texture to and from the device for you, and it even allows you to use priorities etc., as well as flushing the whole on-device (managed) memory. This means that the textures will remain in your process memory, but it seems that you'll still get better memory usage than with your current approach :)
• Possibly, the problems might be related to eg DirectX 9 contexts only. You might want to test with one of the newer interfaces, like DX10 or DXGI. This doesn't necessarily limit you to DX10+ GPUs - but you will lose support for Windows XP (which isn't supported anymore anyway).