C# How to make an in memory object (DataTable) accessible from other applications, asp websites

Question

I basically have a page which needs to have a specific set of data available to it, I also need this data accessible from other applications. SQL is to slow for this, so my first thought is to have a Datatable already populated and use linq to select from it.

How can I have 1 executable house a datatable that I can then access from other applications? Also - is a Datatable the best structure for my data? The data is basically 400k rows by 6 columns.

Answer 1

System.IO.MemoryMappedFiles is what you're looking for. You can initialize your data table and serialize it using WriteXml() . Then map that file to memory by calling System.IO.MemoryMappedFiles.CreateFromFile() method.

Then you could access the file from multiple processes and do stuff with it. Eg you can resemble the data table in another process by calling ReadXml() You can find more info about MemoryMappedFiles here

• This is doable in .NET 4.0 and on!

Answer 2

Another (better, I believe) approach is to employ MarshalByRefObject . This way you could pass the object - actually proxies to the object - between AppDomain s so you don't need serialization at all.

Check out MSDN documentation on the subject

And this is live example I developed couple of years ago, with different goal in mind though. It's a custom web server, which needed to update it self whenever update is available, without interrupting its jobs. So shadow copying was the goal, but I think it is (more) appropriate to use MarshalByRef objects in your scenario.

    public void Boot()
    {
        if (KernelPartition != null)
        {
            throw new InvalidOperationException("Kernel partition already exists.");
        }

        Log.TraceEvent(TraceEventType.Verbose, 0, "Initiating startup sequence...");
        KernelPartition = AppDomain.CreateDomain("Kernel", null, new AppDomainSetup() { ApplicationName = "krnl", ShadowCopyFiles = "true" });
        KernelPartition.DomainUnload += KernelPartition_DomainUnload;

        string engineClassName = null;
        string engineAssemblyName = null;

        try
        {
            var engineTypeId = ConfigurationManager.AppSettings["engineTypeId"];
            engineClassName = engineTypeId.Split(',')[0].Trim();
            engineAssemblyName = engineTypeId.Substring(engineClassName.Length + 1).Trim();
        }
        catch (Exception)
        {
            Log.TraceEvent(TraceEventType.Verbose, 0, "Configuration errors detected. Attempting defaults...");
            engineClassName = "Contoso.Kernel.Turbine";
            engineAssemblyName = "Contoso.Kernel";
        }

        try
        {
            // FOCUS ON THE NEXT LINE
            KernelTurbine = (ITcsKernel)KernelPartition.CreateInstanceAndUnwrap(engineAssemblyName, engineClassName);
            Log.TraceEvent(TraceEventType.Verbose, 0, "Kernel connection established.");
        }
        catch (Exception ex)
        {
            Log.TraceEvent(TraceEventType.Verbose, 0, "Failed to establish communication channel with the kernel with the following exception:\n{0}", ex.ToString());
        }

        if (KernelTurbine == null)
        {
            InitializeRestart();
        }
        else
        {
            try
            {
                Start();
                Log.TraceEvent(TraceEventType.Verbose, 0, "Startup sequence completed.");
                KernelTurbine.RebootDelegate = Reboot;
                DisposeRestartSecond();
            }
            catch (Exception ex)
            {
                string message = string.Format("The startup sequence failed with the following exception:{0}{1}", Environment.NewLine, ex.ToString());
                Log.TraceEvent(TraceEventType.Error, 0, message);

                InitializeRestart();
            }
        }
    }

As I remember, you create the object in the other AppDomain by CreateInstanceAndUnwrap() . This object needs to inherit MarshalByRefObject . So compose a class which inherits it and internally use DataTable .

I hope you can handle it from here. I really can't remember the details but I'm ready to help further should you need.