Passing large amounts of data in an WCF service

Question

I've seen this question pop up a few times but with no real definitive answer (as there is most likely none)... I have a WCF service that needs to return approximately 14,000 rows of data from SQL sorted in an List<> based array.

My service config looks like:

<system.serviceModel>
<bindings>
  <basicHttpBinding>
    <binding name="BasicHttpBinding_IParts" closeTimeout="00:01:00"
        openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
        allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
        maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647"
        messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
        useDefaultWebProxy="true">
      <readerQuotas maxDepth="32" maxStringContentLength="2147483647" maxArrayLength="2147483647"
          maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
      <security mode="None">
        <transport clientCredentialType="None" proxyCredentialType="None"
            realm="" />
        <message clientCredentialType="UserName" algorithmSuite="Default" />
      </security>
    </binding>
  </basicHttpBinding>
</bindings>
<serviceHostingEnvironment multipleSiteBindingsEnabled="true" />
<behaviors>
  <serviceBehaviors>
    <behavior>
      <serviceMetadata httpGetEnabled="true"/>
      <serviceDebug includeExceptionDetailInFaults="true"/>
      <dataContractSerializer maxItemsInObjectGraph="2147483647" />
    </behavior>
  </serviceBehaviors>
</behaviors>

My client config looks like:

<system.serviceModel>
    <bindings>
        <basicHttpBinding>
            <binding name="BasicHttpBinding_IParts" closeTimeout="00:01:00"
                openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
                allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
                maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647"
                messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
                useDefaultWebProxy="true">
                <readerQuotas maxDepth="32" maxStringContentLength="2147483647" maxArrayLength="2147483647"
                    maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
                <security mode="None">
                    <transport clientCredentialType="None" proxyCredentialType="None"
                        realm="" />
                    <message clientCredentialType="UserName" algorithmSuite="Default" />
                </security>
            </binding>
        </basicHttpBinding>
    </bindings>
    <client>
        <endpoint address="http://localhost/RasFuseService/Parts.svc"
            binding="basicHttpBinding" bindingConfiguration="BasicHttpBinding_IParts"
            contract="MyParts.IParts" name="BasicHttpBinding_IParts" />
    </client>
    <behaviors>
      <serviceBehaviors>
        <behavior name="MyServiceBehavior">
          <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
</system.serviceModel>

Something isn't right because i'm getting the error:

The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://tempuri.org/:GetSurplusECMResult . The InnerException message was 'Maximum number of items that can be serialized or deserialized in an object graph is '65536'. Change the object graph or increase the MaxItemsInObjectGraph quota. '. Please see InnerException for more details.

Even though i'm explicitly assigning MaxItemsInObjectGraph with a max int...

I've read about streaming and paging, but is this really too much data to return in a single pass?

Answer 1

Figured it out ( Slapped forehead ) I ended up writing an incorrect entry on the client side. The correct syntax for the client side should have been:

  <behaviors>
    <endpointBehaviors>
      <behavior >
        <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
      </behavior>
    </endpointBehaviors>
  </behaviors>

Not

  <behaviors>
    <serviceBehaviors>
      <behavior>
        <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
      </behavior>
    </serviceBehaviors>
  </behaviors>

It now returns and deserializes my large array without complaining...

Answer 2

I had a case where my WCF service returned raw tabular data, a few megabytes at a time.

The XML serialization was bad... lots of elements and wasted space, and also slow. I handled my own response generation in this case and returned data formatted as CSV data. It was considerably faster, both to generate, send, and parse by the client.

I was using REStful WCF (this was before ASP.NET replaced WCF for REST), so I just returned a Stream from my WCF method.

Answer 3

In your client config your behaviour has a name, but you don't seem to use that name. Either use the name somewhere or omit the name to make your behaviour the default behaviour as in the service config.