WCF CommunicationException related to ServiceKnownType after running for sometime (or IIS App Pool recycle?)

Question

I have a WCF service, hosted in IIS. This service is defined by a generic interface with interface-types as arguments or return types, and so we make use of the ServiceKnownType attribute to define the available implementations of said interfaces at runtime.

This all seems to work fine, however on occasion, we see all requests to these services fail with CommunicationException; " There was an error while trying to serialize parameter http://tempuri.org/ : arg . The InnerException message was 'Type ' MyNamespace.SomeInterfaceImplementation ' with data contract name 'SomeInterfaceImplementation:http://schemas.datacontract.org/2004/07/MyNamespace' is not expected. Add any types not known statically to the list of known types - for example, by using the KnownTypeAttribute attribute or by adding them to the list of known types passed to DataContractSerializer.'. Please see InnerException for more details. "

I am unable to reliably reproduce this error, however it regularly appears after the services have been left running for some time (eg; over the weekend). I initially speculated that this was occurring due to IIS App Pool recycling, however manual recycling, or scheduled recycling on small intervals (eg; 2 minutes) fails to reproduce the problem.

I am able to reliably reproduce the exception by excluding ' MyNamespace.MyType ' from the provider of 'ServiceKnownTypes', however that doesn't really tell me why this is occurring intermittently in the first place.

The following snippet shows the service declaration. It is a generic service for different implementation types of . Note that it is the operation argument ' arg ' which is producing the CommunicationException.

[ServiceKnownType("GetKnownTypes", typeof(KnownTypesCache))]
[ServiceContract]
public interface IMyService<T>
    where T : class, IMyServiceTypeInterface
{
    [OperationContract]
    [TransactionFlow(TransactionFlowOption.Allowed)]
    MyReturnType HandleRequest(T element, ISomeInterfaceArgumentType arg);
}

Now, the implementation of KnownTypesCache, which provides the known types for ISomeInterfaceArgumentType is like so;

public static class KnownTypesCache
{
    private static readonly List<Assembly> queriedAssemblies = new List<Assembly>();
    private static readonly List<Type> knownTypes = new List<Type>();


    static KnownTypesCache()
    {
        // get all available assemblies at this time
        List<Assembly> assemblies = AppDomain.CurrentDomain.GetAssemblies().ToList();

        // find all available known types publishers
        IEnumerable<Type> knownTypesPublisherTypes = assemblies
            .Where(a => !queriedAssemblies.Contains(a)) // exclude already queried assemblies to speed things up
            .SelectMany(s => s.GetTypes())
            .Where(p => typeof(IKnownTypesPublisher).IsAssignableFrom(p) && p.HasAttribute(typeof(KnownTypesPublisherAttribute)));

        // add all known types
        foreach (Type type in knownTypesPublisherTypes)
        {
            IKnownTypesPublisher publisher = (IKnownTypesPublisher)Activator.CreateInstance(type);

                AddRange(publisher.GetKnownTypes());
            }
        }


        // record the assemblies we've already loaded to avoid relookup
        queriedAssemblies.AddRange(assemblies);
    }

    public static IEnumerable<Type> GetKnownTypes(ICustomAttributeProvider provider)
    {
        return knownTypes;
    }
}

Basically, this global static cache queries all loaded assemblies in the AppDomain for implementors of 'IKnownTypesPublisher', which in turn provide available serialization types for each assembly. So, there is one IKnownTypesPublisher per assembly, which is responsible for identifying types in that assembly only, and the KnownTypesCache simply aggregates all of these and returns to the DataContractSerializer at runtime.

As I mentioned, this approach seems to work fine 99% of the time. And then for no reason I can identify, it stops working and can only be resolved by a call to iisreset .

I am pretty stumped now, I have tried all sorts of solutions, but since I can only reliably reproduce this error by waiting until first thing Monday, it's a bit tough!

My last remaining thought is that the static KnownTypesCache constructor might be being called before all assemblies are loaded to the AppDomain, and so the cache will be empty for the life of the instance...?

Answer 1

My last remaining thought is that the static KnownTypesCache constructor might be being called before all assemblies are loaded to the AppDomain

With this, you are probably well on the way to answering your own question.

AppDomain.GetAssemblies() only returns those assemblies which have already been loaded into the execution context of the AppDomain. After any recycling of the IIS worker process, which creates a new AppDomain, you have a potential race condition between the first use of the KnownTypesCache type, and the loading of any assembly containing one of your IKnownTypesPublisher types. Intermittent, hard-to-reproduce bugs, can frequently be caused by race conditions.

For your design to work correctly, you would need to ensure somehow that the service implementation always loads all the assemblies containing your known types, before anything calls a KnownTypesCache method.