How can I improve the speed of loading xml documents

Question

I need to load 2 types of xml documents; one has 50 sub-children and the other has the same 50 and 800 additional ones. Performance is great with the smaller doc and acceptable with the larger doc until the number of children increases. 20k children * 50 sub-children = great performance, 20k children * 850 sub-children = slow performance. How would I skip looking for the extra descendants when they do not exist? My initial attempts lead me to think I need have separate classes, methods, viewmodels, and views for both the small and large docs. Below is a condensed look at my code.

public class MyItem
    {
        private string layout;
        private string column;
        private string columnSpan;
        private string row;
        private string rowSpan;
        private string background;

public MyItem(string layout, string column, string columnSpan, string row, string rowSpan, string background)
{
        Layout = layout;
        Column = column;
        ColumnSpan = columnSpan;
        Row = row;
        RowSpan = rowSpan;
        Background = background;
}

public string Layout
    {
        get { return this.layout; }
        set { this.layout = value; }
    }

(Not Shown - Column, ColumnSpan, Row, RowSpan, and Background which are handled the same way as Layout)

Just for this example, below shows only 6 sub-children, I am looking for a way to load xml docs with only the first 2 sub-children. This way I can use whatever load method is required by both small or large xml docs.

internal class myDataSource
{
    //Loads (MyList) xml file 

    public static List<MyItem> Load(string MyListFilename)
    {

        var myfiles = XDocument.Load(MylistFilename).Descendants("item").Select(
            x => new MyItem(
                (string)x.Element("layout"),
                (string)x.Element("column"),
                (string)x.Element("columnSpan"),
                (string)x.Element("row"),
                (string)x.Element("rowSpan"),
                (string)x.Element("background")));

    return myfiles.ToList();
    }

public class MainViewModel : ViewModelBase
{

public void LoadMyList()
        {   

            this.myfiles = new ObservableCollection<MyItemViewModel>();

            List<MyItem> mybaseList = myDataSource.Load(MyListFilename);

            foreach (MyItem myitem in mybaseList)
            {
                this.myfiles.Add(new MyItemViewModel(myitem));
            }


            this.mycollectionView = (ICollectionView)CollectionViewSource.GetDefaultView(myfiles);
            if (this.mycollectionView == null)
                throw new NullReferenceException("mycollectionView");                
        }
}

 public class MyItemViewModel: ViewModelBase
{

    private Models.MyItem myitem;


    public MyItemViewModel(MyItem myitem)
    {
        if (myitem == null)
            throw new NullReferenceException("myitem");

        this.myitem = myitem;
    }      


    public string Layout
    {
        get
        {
            return this.myitem.Layout;
        }
        set
        {
            this.myitem.Layout = value;
            OnPropertyChanged("Layout");
        }
    }

(Not Shown - Column, ColumnSpan, Row, RowSpan, and Background which are handled the same way as Layout)

Answer 1

Instead of using Descendants , can you follow the direct path (ie use Elements )? That's the only way you'll keep from scanning nodes you know don't have items.

Answer 2

i think one thing you can do is not do a toList on the Select and keep it as lazy, and return an Iterable instead, or whatever Select returns.(sorry, i don't have a windows box right now to test this on). when you do the foreach, you will iterate over it only once (instead of twice right now)

Answer 3

XDocument is handy, but if the problem is simply that the files are large and you only have to scan once through, XmlReader might be the better choice. It doesn't read the entire file, it reads one node at a time. You can manually skip through parts you are not interested in.