XML序列化：64位Windows7，oct处理器内核，32GB RAM和OutOfMemory异常

Question

While processing 500 MB data and trying to serialize it,

GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)

throws System.OutOfMemoryException

XML Serializer:

StringWriter xmlStr = new StringWriter();

XmlTextWriter twa = new XmlTextWriter(xmlStr);

lock(myLock)
{
    mySerializer.WriteObject(twa, myHandlerMap); // OutOfMemoryException

    twa.Flush();
    twa.Close();

    set.StringBuffer = xmlStr.ToString();

    xmlStr.Dispose()
}

After reading the last paragraph at following link: http://blogs.msdn.com/b/vsofficedeveloper/archive/2008/10/10/stringbuilder-outofmemoryexception.aspx

I doubt if it is possible to solve this issue without MS fixing it?

Exception Trace:

RemoteStackTrace:    
   at System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)
   at System.Text.StringBuilder.GetNewString(String currentString, Int32 requiredLength)
   at System.Text.StringBuilder.Append(Char value)
   at System.IO.StringWriter.Write(Char value)
   at System.Xml.XmlTextWriter.InternalWriteEndElement(Boolean longFormat)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContextComplex.WriteString(XmlWriterDelegator xmlWriter, String value, XmlDictionaryString name, XmlDictionaryString ns)
   at WriteKeyValueOfstringstringToXml(XmlWriterDelegator , Object , XmlObjectSerializerWriteContext , ClassDataContract )
   at System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator xmlWriter, Object obj, XmlObjectSerializerWriteContext context)
   at WriteArrayOfKeyValueOfstringstringToXml(XmlWriterDelegator , Object , XmlObjectSerializerWriteContext , CollectionDataContract )
   at System.Runtime.Serialization.CollectionDataContract.WriteXmlValue(XmlWriterDelegator xmlWriter, Object obj, XmlObjectSerializerWriteContext context)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeAndVerifyType(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, Boolean verifyKnownType, RuntimeTypeHandle declaredTypeHandle)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithXsiType(XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle objectTypeHandle, Type objectType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle, Type declaredType)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContextComplex.InternalSerializeWithSurrogate(XmlWriterDelegator xmlWriter, Object obj, Boolean isDeclaredType, Boolean writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(XmlWriterDelegator xmlWriter, Object obj, Boolean isDeclaredType, Boolean writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle)
   at WriteGenericViewingMPStateToXml(XmlWriterDelegator , Object , XmlObjectSerializerWriteContext , ClassDataContract )
   at System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator xmlWriter, Object obj, XmlObjectSerializerWriteContext context)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeAndVerifyType(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, Boolean verifyKnownType, RuntimeTypeHandle declaredTypeHandle)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithXsiType(XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle objectTypeHandle, Type objectType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle, Type declaredType)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContextComplex.InternalSerializeWithSurrogate(XmlWriterDelegator xmlWriter, Object obj, Boolean isDeclaredType, Boolean writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(XmlWriterDelegator xmlWriter, Object obj, Boolean isDeclaredType, Boolean writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteSerializationInfo(XmlWriterDelegator xmlWriter, Type objType, SerializationInfo serInfo)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteISerializable(XmlWriterDelegator xmlWriter, ISerializable obj)
   at System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator xmlWriter, Object obj, XmlObjectSerializerWriteContext context)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContextComplex.InternalSerializeWithSurrogate(XmlWriterDelegator xmlWriter, Object obj, Boolean isDeclaredType, Boolean writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(XmlWriterDelegator xmlWriter, Object obj, Boolean isDeclaredType, Boolean writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle declaredTypeHandle)
   at WriteKeyValueOfstringStateSerializeSetjJgp3TWnToXml(XmlWriterDelegator , Object , XmlObjectSerializerWriteContext , ClassDataContract )
   at System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator xmlWriter, Object obj, XmlObjectSerializerWriteContext context)
   at WriteArrayOfKeyValueOfstringStateSerializeSetjJgp3TWnToXml(XmlWriterDelegator , Object , XmlObjectSerializerWriteContext , CollectionDataContract )
   at System.Runtime.Serialization.CollectionDataContract.WriteXmlValue(XmlWriterDelegator xmlWriter, Object obj, XmlObjectSerializerWriteContext context)
   at System.Runtime.Serialization.DataContractSerializer.InternalWriteObjectContent(XmlWriterDelegator writer, Object graph)
   at System.Runtime.Serialization.DataContractSerializer.InternalWriteObject(XmlWriterDelegator writer, Object graph)
   at System.Runtime.Serialization.XmlObjectSerializer.WriteObjectHandleExceptions(XmlWriterDelegator writer, Object graph)

Pls. let me know how can it be solved?

Answer 1

500MB is pretty big for xml, and it would not surprise me to hear that it isn't happy. If possible, I would look at something streaming -based, ie XmlWriter / XmlReader . Without more detail it isn't clear whether that is possible here. Alternatively, consider another format! Seriously... xml doesn't play nicely at big sizes.

The first thing to try, though, is... don't use a string (or StringBuilder); but instead, read/write directly from/to disk via a FileStream over an XmlReader / XmlWriter .

Answer 2

I would prefer serializing this directly into a file. 500 MB is a quite a huge data and will getting much more when serializing it in "chatty" xml.