How to avoid memory leak using Roslyn CSharpCompilation

Question

First of all, I am using Windows 10 with visual studio 2019 using .net core 3.1 I am using the Roslyn from Nuget Microsoft.Net.Compilers 3.4.0 and Microsoft.CodeAnalysis 3.4.0

I am using Roslyn in my project to dynamically compile scripts. I noticed that the memory usage of my application was astronomically high(20-50MB of ram for every scripts) for compiling a few very small scripts (5-10kb).

I first thought that the the memory usage was coming from loading the assemblies, but I isolated the CSharpCompilation.Create that seems to take a good 10MB while CSharpCompilation.Emit take the rest (10-40MB).

I reduced it to reproduce the bug

public byte[] CompileCSharpCode(string assemblyName, string code)
{
    SyntaxTree syntaxTree = SyntaxFactory.ParseSyntaxTree(code, null, "");
    CSharpCompilation compilation = CSharpCompilation.Create
    (
        assemblyName,
        new[] { syntaxTree },
        options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary),
        references: _referencesList
    );
    //at this point the memory usage goes up by 10MB
    using (var stream = new MemoryStream())
    {
        var emitResult = compilation.Emit(stream);
        //at this point the memory usage goes up by 10-40MB
        return stream.ToArray();
    }
}

I expect the memory to go up when the the functions are called, objects are created and compilation happens, but the memory never get release. After compiling 200 scripts, the memory usage is several GB.

The memory stays high even if I get rid of every object that was used for the compilation, including the byte array that hold the result.

At the moment I resorted to create a second .net core application that I call to precompile every scripts 1 by 1 that I write to .dll files and load them in memory (which takes about 20MB total in memory for the 200 scripts)

Is there a way to compile my script with Roslyn without having this huge memory issue?

Answer 1

The problem is that Roslyn is creating and loading an assembly for your compilation and it never gets unloaded. Use the new collectible AssemblyLoadContext feature of .NET Core 3.0 so that you can unload it. I wrote this article to demonstrate it.