Fast changing collection MVVM WPF - high CPU usage & UI almost freezes

Question

I am developing an app with a datagrid of that displays certain running Windows processes (in my example Chrome processes). The datagrid is loaded with processes when a checkbox is checked.

Requirements:

• Display 'live' info for the name, memory usage (private working set) of each process, just like in the Windows Task Manager - Processes tab.
• Monitor for processes that exit and remove them from the datagrid.
• Monitor for certain processes that start.

Used techniques:

• MVVM
• MVVM Light
• Benoît Blanchon approach for fast changing properties
• Thomas Levesque AsyncObservableCollection to modify an ObservableCollection from another thread

Issue(s):

• When the processes are loaded, the CPU usage gets very high and the UI almost freezes.
• CPU usage remains high even when the ManagerService.Stop() is called.
• Sometimes a System.InvalidOperationException - Cannot change ObservableCollection during a CollectionChanged event exception is thrown when a process is removed from the collection.

How can I fix this issues? Also is my approach a 'good practice' one?

Any help would be greatly appreciated! I've already spent a lot of time on this issue.

Update 1

Didn't help, removing OnRendering() and implementing INotifyPropertyChanged

public class CustomProcess : INotifyPropertyChanged
{
    private double _memory;

    public double Memory
    {
        get { return _memory; }
        set
        {
            if (_memory != value)
            {
                _memory = value;
                OnPropertyChanged(nameof(Memory));
            }
        }
    }


    private bool _isChecked;

    public bool IsChecked
    {
        get { return _isChecked; }
        set
        {
            if (_isChecked != value)
            {
                _isChecked = value;
                OnPropertyChanged(nameof(IsChecked));
    }
}

Update 2

Following Evk advice I've updated

• Used regular ObservableCollection
• moved timer to viewmodel

CPU usage is much lower now. However I sometimes get an Process with an ID of ... is not running exception in the OnProcessStarted()

Viewmodel

public class MainViewModel 
    {
        System.Threading.Timer timer;
        private ObservableCollection<CustomProcess> _processes;
        public ObservableCollection<CustomProcess> Processes
        {
            get
            {
                if (_processes == null)
                    _processes = new ObservableCollection<CustomProcess>();

                return _processes;
            }
        }
        private void OnBooleanChanged(PropertyChangedMessage<bool> propChangedMessage)
        {
            if (propChangedMessage.NewValue == true)
            {
                _managerService.Start(_processes);
                timer = new System.Threading.Timer(OnTimerTick, null, 0, 200); //every 200ms
                ProcessesIsVisible = true;
            }
            else
            {
                timer.Dispose();
                _managerService.Stop();
                ProcessesIsVisible = false;
            }
        }
        private void OnTimerTick(object state)
        {
            try
            {
                for (int i = 0; i < Processes.Count; i++)
                    Processes[i].UpdateMemory();
            }
            catch (Exception)
            {

            }
        }

Model

public class CustomProcess : INotifyPropertyChanged
    {    
        public void UpdateMemory()
        {
            if (!ProcessObject.HasExited)
                Memory = Process.GetProcessById(ProcessObject.Id).PagedMemorySize64;
        }
        private double _memory;

        public double Memory
        {
            get { return _memory; }
            set
            {
                if (_memory != value)
                {
                    _memory = value;
                    OnPropertyChanged(nameof(Memory));
                }
            }
        }

Service

        private void OnProcessNotification(NotificationMessage<Process> notMessage)
        {
            if (notMessage.Notification == "exited")
            {
                _processes.Remove(p => p.ProcessObject.Id == notMessage.Content.Id, DispatcherHelper.UIDispatcher);
            }

        }

Original code

XAML

<DataGrid ItemsSource="{Binding Processes}">
   <DataGridTextColumn Header="Process name"
                            Binding="{Binding ProcessObject.ProcessName}"
                            IsReadOnly='True'
                            Width='Auto' />
        <DataGridTextColumn Header="PID"
                            Binding="{Binding ProcessObject.Id}"
                            IsReadOnly='True'
                            Width='Auto' />
        <DataGridTextColumn Header="Memory"
                            Binding='{Binding Memory}'
                            IsReadOnly='True'
                            Width='Auto' />
</DataGrid>

XAML Code behind

public MainWindow()
{
        InitializeComponent();
        DataContext = SimpleIoc.Default.GetInstance<MainViewModel>();
        CompositionTarget.Rendering += OnRendering;
    }

    private void OnRendering(object sender, EventArgs e)
    {
        if (DataContext is IRefresh)
            ((IRefresh)DataContext).Refresh();
    }
}

ViewModel

public class MainViewModel : Shared.ViewModelBase, IRefresh
{
    private AsyncObservableCollection<CustomProcess> _processes;
    public AsyncObservableCollection<CustomProcess> Processes
    {
        get
        {
            if (_processes == null)
                _processes = new AsyncObservableCollection<CustomProcess>();

            return _processes;
        }
    }
    private readonly IManagerService _managerService;

    public MainViewModel(IManagerService managerService)
    {
        _managerService = managerService;
        Messenger.Default.Register<PropertyChangedMessage<bool>>(this, OnBooleanChanged);
    }      

    #region PropertyChangedMessage
    private void OnBooleanChanged(PropertyChangedMessage<bool> propChangedMessage)
    {
        if (propChangedMessage.NewValue == true)
        {
            _managerService.Start(_processes);
        }
        else
        {
            _managerService.Stop();
        }
    }

    public void Refresh()
    {
        foreach (var process in Processes)
            RaisePropertyChanged(nameof(process.Memory)); //notify UI that the property has changed
    }

Service

public class ManagerService : IManagerService
{
    AsyncObservableCollection<CustomProcess> _processes;
    ManagementEventWatcher managementEventWatcher;

    public ManagerService()
    {
        Messenger.Default.Register<NotificationMessage<Process>>(this, OnProcessNotification);
    }

    private void OnProcessNotification(NotificationMessage<Process> notMessage)
    {
        if (notMessage.Notification == "exited")
        {
            //a process has exited. Remove it from the collection
            _processes.Remove(p => p.ProcessObject.Id == notMessage.Content.Id);
        }

    }

    /// <summary>
    /// Starts the manager. Add processes and monitor for starting processes
    /// </summary>
    /// <param name="processes"></param>
    public void Start(AsyncObservableCollection<CustomProcess> processes)
    {
        _processes = processes;
        _processes.CollectionChanged += OnCollectionChanged;

        foreach (var process in Process.GetProcesses().Where(p => p.ProcessName.Contains("chrome")))
            _processes.Add(new CustomProcess(process));

        MonitorStartedProcess();
        Task.Factory.StartNew(() => MonitorLogFile());
    }

    /// <summary>
    /// Stops the manager.
    /// </summary>
    public void Stop()
    {       
        _processes.CollectionChanged -= OnCollectionChanged;
        managementEventWatcher = null;
        _processes = null;
    }

    private void MonitorLogFile()
    {
        //this code monitors a log file for changes. It is possible that the IsChecked property of a CustomProcess object is set in the Processes collection
    }

    /// <summary>
    /// Monitor for started Chrome
    /// </summary>
    private void MonitorStartedProcess()
    {
        var qStart = "SELECT * FROM Win32_ProcessStartTrace WHERE ProcessName like '%chrome%'";
        ManagementEventWatcher managementEventWatcher = new ManagementEventWatcher(new WqlEventQuery(qStart));
        managementEventWatcher.EventArrived += new EventArrivedEventHandler(OnProcessStarted);
        try
        {
            managementEventWatcher.Start();
        }
        catch (Exception)
        {

        }
    }



    private void OnProcessStarted(object sender, EventArrivedEventArgs e)
    {

        try
        {
            int pid = Convert.ToInt32(e.NewEvent.Properties["ProcessID"].Value);
            _processes.Add(new CustomProcess(Process.GetProcessById(pid)));  //add to collection
        }
        catch (Exception)
        {

        }

    }

Model

public class CustomProcess
{        
    public Process ProcessObject { get; }

    public CustomProcess(Process process)
    {
        ProcessObject = process;
        try
        {
            ProcessObject.EnableRaisingEvents = true;
            ProcessObject.Exited += ProcessObject_Exited;
            Task.Factory.StartNew(() => UpdateMemory());
        }
        catch (Exception)
        {

        }

    }

    private void ProcessObject_Exited(object sender, EventArgs e)
    {
        Process process = sender as Process;
        NotificationMessage<Process> notMessage = new NotificationMessage<Process>(process, "exited");
        Messenger.Default.Send(notMessage); //send a notification that the process has exited
    }

    private void UpdateMemory()
    {
        while (!ProcessObject.HasExited)
        {
            try
            {
                Memory = Process.GetProcessById(ProcessObject.Id).PagedMemorySize64;
            }
            catch (Exception)
            {

            }
        }
    }

    private double _memory;

    public double Memory
    {
        get { return _memory; }
        set
        {
            if (_memory != value)
            {
                _memory = value;
            }
        }
    }


    private bool _isChecked;

    public bool IsChecked
    {
        get { return _isChecked; }
        set
        {
            if (_isChecked != value)
            {
                _isChecked = value;
            }
        }
    }

Answer 1

Writing to a GUI is expensive. If you only do it once per user triggered event you will not notice it. But once you write from any kind of loop - including a loop running on another thread - you will notice it. I even wrote some example code for Windows Forms to showcase this:

using System;
using System.Windows.Forms;

namespace UIWriteOverhead
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        int[] getNumbers(int upperLimit)
        {
            int[] ReturnValue = new int[upperLimit];

            for (int i = 0; i < ReturnValue.Length; i++)
                ReturnValue[i] = i;

            return ReturnValue;
        }

        void printWithBuffer(int[] Values)
        {
            textBox1.Text = "";
            string buffer = "";

            foreach (int Number in Values)
                buffer += Number.ToString() + Environment.NewLine;
            textBox1.Text = buffer;
        }

        void printDirectly(int[] Values){
            textBox1.Text = "";

            foreach (int Number in Values)
                textBox1.Text += Number.ToString() + Environment.NewLine;
        }

        private void btnPrintBuffer_Click(object sender, EventArgs e)
        {
            MessageBox.Show("Generating Numbers");
            int[] temp = getNumbers(10000);
            MessageBox.Show("Printing with buffer");
            printWithBuffer(temp);
            MessageBox.Show("Printing done");
        }

        private void btnPrintDirect_Click(object sender, EventArgs e)
        {
            MessageBox.Show("Generating Numbers");
            int[] temp = getNumbers(1000);
            MessageBox.Show("Printing directly");
            printDirectly(temp);
            MessageBox.Show("Printing done");
        }
    }
}

Your code is even slightly worse, as you allow the Update and thus Layout code to run between each update. While it does keep the UI responsive, it is more code to run.

You will not get around limiting the updates. I would put these kinds of Limitations clearly on the View Side. Personally I prefer this way:

1. Do not register the Change Notificaiton events realted to the Observable collection
2. Make a timer that regularly updates the UI with the current value of the Collection. Set the timer to something like 60 Updates per second. That should be fast enough for humans.
3. You may want to add some form of Locking to the code writing the Collection and the accessor code to avoid race conditions.

A few side notes:

A pet Peeve of mine is Exception Hanlding. And I see some swallowing of Fatal Exceptions there. You really should fix that ASAP. It is bad enough that Threads can accidentally swallow exceptions, you should not write additional code for this. Here are two articles I link a lot: http://blogs.msdn.com/b/ericlippert/archive/2008/09/10/vexing-exceptions.aspx | http://www.codeproject.com/Articles/9538/Exception-Handling-Best-Practices-in-NET

Secondly, ObservableColelctions are notoriously bad with complete reworks. It lacks a add-range Function. So every single change will trigger an update. My usual workaround is: 1. Give the property exposing the Collection Change Notification 2. Do not work with the exposed collection on any update. 3. Instead work with a background collection. Only when this new state is finished, do you expose it.

Answer 2

Instead of you updating/refresing the UI yourself, make use of the WPF change notification system achieved using DataBinding & PropertyChanged event.

As MSDN quotes -

The INotifyPropertyChanged interface is used to notify clients, typically binding clients, that a property value has changed.

For example, consider a Person object with a property called FirstName . To provide generic property-change notification, the Person type implements the INotifyPropertyChanged interface and raises a PropertyChanged event when FirstName is changed.

More details here .