XDocument.Load ( XmlReader) has horrible performance; 2 MB XML from Web-Service needs 4 sec from Stream to parse

Question

I am calling a webservice, which returns a xml with about 2MB.

Everything quite standard. The Problem is the creation of a XDocument.

XDocument xdoc = XDocument.Load( XMLReader Object); // takes 4 sec!!!

I am creating xdoc because I use LINQ to XML to read the XML.

Stopwatch s = new Stopwatch();

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(MyUri);
System.Net.ServicePointManager.Expect100Continue = false;
req.Method = "POST";

req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = Poststring.Length;

StreamWriter swriter = new StreamWriter(req.GetRequestStream());
swriter.Write(Poststring);
swriter.Close();

s.Start();
HttpWebResponse response = (HttpWebResponse)req.GetResponse();
XmlReader MyXmlReader = null;

XmlReaderSettings settings = new XmlReaderSettings();
settings.ProhibitDtd = false;
settings.ValidationType = ValidationType.None;
settings.ConformanceLevel = ConformanceLevel.Document;

Stream stream = response.GetResponseStream();
s.Stop(); 
Debug.WriteLine("stream= response.GetResponseStream(): " + s.ElapsedMilliseconds);
s.Reset();

MyXmlReader = XmlReader.Create(stream, settings);

Debug.WriteLine("Before XDocument.Load(MyXmlReader): " + s.ElapsedMilliseconds);
s.Start();
XDocument xdoc = XDocument.Load(MyXmlReader);
s.Stop();
Debug.WriteLine("Duration: " + s.ElapsedMilliseconds);

The PC is 2 years old, 4 GB RAM. The PC is ok. I tested it on 2 other PCs and Laptop and got same results. My way of the creation of XDocument xdoc takes just to long, but why?

I tested

XDocument xdoc = XDocument.Load(String with path to the same xmlFile on my pc);

and it took like 20 Milliseconds.

EDIT: here some timings:

stream= response.GetResponseStream(): 5276 Milliseconds

I guess this is the time from Server to my pc

Duration: 4855

time needed for: XDocument xdoc = XDocument.Load(MyXmlReader);

Maybe a problem because it is Stream beeing "converted" into XDocument xdoc object?

edit2: I just tested

HttpWebResponse response = (HttpWebResponse)req.GetResponse();
MemoryStream ms = new MemoryStream();

//responsetime from server to my pc: 6000 ms
Stream stream = response.GetResponseStream(); 
stream.CopyTo(ms); //this operation takes 4000 ms!!!
ms.Position = 0;
XDocument x4 = XDocument.Load(ms);// this takes 13 ms!!!

The conversion of the data in the stream object to some useful data(like string or ms) take that long, but why?

Am I right, that all data from Web-service are 100% sent and have arrived and only then the next line of code is beeing performed? or is stream.copyTo(ms) operating on a stream that is open and still receiving data?

Answer 1

In the line:

MyXmlReader = XmlReader.Create(stream, settings);

You are reading from a Stream , which is essentially a pipe , not a bucket ; the stream does not yet have all that data. I'm guessing that 2MB is taking about 4 seconds to arrive down the wire.

If that is too long, make sure you have enabled gzip/deflate on the http connection. As oberfreak notes, there are other data formats that are more suited to large documents (xml can become unwieldy at large sizes, although it will generally work).