Creating Large List<T>

Question

I have the following code:

var lstMusicInfo = new List<MediaFile>();
var LocalMusic = Directory.EnumerateFiles(AppSettings.Default.ComputerMusicFolder, "*.*", SearchOption.AllDirectories).AsParallel().ToList<string>();
LocalMusic = (from a in LocalMusic.AsParallel()
              where a.EndsWith(".mp3") || a.EndsWith(".wma")
              select a).ToList<string>();
var DeviceMusic = adb.SyncMedia(this.dev, AppSettings.Default.ComputerMusicFolder, 1);

---

Parallel.ForEach(LocalMusic, new Action<string>(item =>
    {
        try
        {
            UltraID3 mFile = new UltraID3();
            FileInfo fInfo;
            mFile.Read(item);
            fInfo = new FileInfo(item);
            bool onDevice = true;
            if (DeviceMusic.Contains(item))
            {
                onDevice = false;
            }
            // My Problem starts here
            lstMusicInfo.Add(new MediaFile()
            {
                Title = mFile.Title,
                Album = mFile.Album,
                Year = mFile.Year.ToString(),
                ComDirectory = fInfo.Directory.FullName,
                FileFullName = fInfo.FullName,
                Artist = mFile.Artist,
                OnDevice = onDevice,
                PTDevice = false
            });
            //Ends here.
        }
        catch (Exception) { }
    }));
    this.Dispatcher.BeginInvoke(new Action(() =>
    {
        lstViewMusicFiles.ItemsSource = lstMusicInfo;
        blkMusicStatus.Text = "";
        doneLoading = true;
    }));
    #endregion
}));

---

The first part of the code gives me almost instant result containing:

1. Address on computer of 5780 files.

2. Get list of all music files on an android device compare it with those 5780 files and return a list of files found on computer but not on device (in my case it returns a list with 5118 items).

---

The block of code below is my problem, I am filling data into a class, then adding that class into a List<T> , doing it for 5780 times takes 60 seconds, how can I improve it?

// My Problem starts here
lstMusicInfo.Add(new MediaFile
    {
        Title = mFile.Title,
        Album = mFile.Album,
        Year = mFile.Year.ToString(),
        ComDirectory = fInfo.Directory.FullName,
        FileFullName = fInfo.FullName,
        Artist = mFile.Artist,
        OnDevice = onDevice,
        PTDevice = false
    });
//Ends here.

Update: Here is the profiling result and I see it's obvious why it's slowing down >_> I suppose I should look for a different library that reads music file information.

Answer 1

One way to avoid loading everything once, up front, would be to lazy load the ID3 information as necessary.

You'd construct your MediaFile instances thus...

new MediaFile(filePath)

...and MediaFile would look something like the following.

internal sealed class MediaFile
{
    private readonly Lazy<UltraID3> _lazyFile;

    public MediaFile(string filePath)
    { 
        _lazyFile = new Lazy<UltraID3>(() =>
        {
            var file = new UltraID3();
            file.Read(filePath);
            return file;
        });
    }

    public string Title
    {
        get { return _lazyFile.Value.Title; }
    }

    // ...
}

This is possibly less ideal than loading them as fast as you can in the background, if you do something like MediaFiles.OrderBy(x => x.Title).ToList() and nothing has been lazy loaded then you'll have to wait for every file to load.

Loading them in the background would make them available for use immediately after the background loading has finished. But you might have to concern yourself with not accessing some items until the background loading has finished.

Answer 2

You biggest bottleneck is new FileInfo(item) , but you don't need FileInfo just to get the Directory and File names. You can use Path.GetDirectoryName and Path.GetFileName , which are must faster since no I/O is involved.

                UltraID3 mFile = new UltraID3();
                //FileInfo fInfo;
                mFile.Read(item);
                //fInfo = new FileInfo(item);
                bool onDevice = true;
                if (DeviceMusic.Contains(item))
                {
                    onDevice = false;
                }
                // My Problem starts here
                lstMusicInfo.Add(new MediaFile()
                {
                    Title = mFile.Title,
                    Album = mFile.Album,
                    Year = mFile.Year.ToString(),
                    ComDirectory = Path.GetDirectoryName(item), // fInfo.Directory.FullName,
                    FileFullName = Path.GetFileName(item), //fInfo.FullName,
                    Artist = mFile.Artist,
                    OnDevice = onDevice,
                    PTDevice = false
                });
                //Ends here.