NancyFx + SignalR + Async progress reporting using Progress<T>

Question

I turn to you in semi-despiration... I have been trying to build some progress reporting into a NancyFx app using SignalR, Async in .net 4.5 and Progress. My main inspiration comes from this article: https://blog.safaribooksonline.com/2014/02/06/server-side-signalr/

I have customized the code a bit to make it also support progress step reporting, not just percentages.

The problem is this: I have a heavy duty task called ReviewSetup.SetupAsync(.., ..., IProgress progress), called from the Nancy module using the ProgressJobManager.Instance.DoJobAsync method. I pass in the IProgress as a final parameter of the SetupAsync method. I setup an EventHandler for Progress.ProgressChanged which should be called every time I report some progress from within the SetupAsync() method.

As you can see in my code below, I wire up the event handler in the ProgressJobManager.Instance.DoJobAsync() method, which is probably one of the places things start to go wrong, ExecutionContext-wise/thread-wise.

The EventHandler for the ProgressChanged event of Progress, in turn, updates the ProgressJob by calling the ReportStep() method. These methods are never called, in the sense that they never break on any break point I set, which I think is also due to the SetupAsync() method being executed on a different thread, and thus not being able to report back using the provided parameter IProgress progress...

This problem of the progress reports not being called properly cascades down to the ProgressJob.StepAdded event not being raised, thus not being catched by the JobManger, thus not being able to tell the ProgressHub to send a message to the clients, notifying them of the progress update.

The article mentioned above requires all the heavy duty work (and thus the reporting back of progress) to be done in the ProgressJobManager.Instance.DoJobAsync() method. This is fine, but I want to call the ReviewSetup.SetupAsync() method and to have THAT method report the progress ... I hope you guys are still with me :-). I really want to get my head around this async in .net 4.5 stuff. I might have aimed a bit high by immidiatly wanting to build it into a more complex application, but that is between me and my ego :-).

Any way, my questions are:

• Have I approached this in completely the wrong way?
• Is it an ExecutionContext/thread issue, that due to my light experience in all things async/await is easily fixed by better understanding of the workings of async in .net 4.5? And if so, please give me pointers in the right direction! :-)

Sorry, this is not your garden variety "how do I convert to type of question. Hence, you help is even more appreciated then usual! :-)

Here is my code:

ReviewSetup.cs (the class that does the actual reporting):

public class ReviewSetup
{

//other stuff here

    public async Task SetupAsync(ReviewSetupConfiguration setupConfiguration, string    notebookGuid, List<string> tagGuids, ReflectUser user, string reviewName, IProgress<ProgressStatus> progress)
    {
        NoteFilter = notebookGuid != "0" ? new NoteFilter() { NotebookGuid = notebookGuid, TagGuids = tagGuids } : new NoteFilter() { TagGuids = tagGuids };
        Name = reviewName;
        _user = user;
        UserId = user.Id;
        ReviewSetupConfiguration = setupConfiguration;
        await Task.Run(() =>
        {
            progress.Report(new ProgressStatus() { Step = "Searching for notes. This might take a few moments..." });
            GetBaseNoteList();
            progress.Report(new ProgressStatus() { Step = BaseNoteList.Notes.Count + " notes found." });
            if (NoteFilter.NotebookGuid != null)
            {
                progress.Report(new ProgressStatus() { Step = "Getting notebook name..." });
                GetNotebookName();
                progress.Report(new ProgressStatus() { Step = "Notebook name found: " + NotebookName });
            }
            if (NoteFilter.TagGuids != null)
            {
                progress.Report(new ProgressStatus() { Step = "Getting tag names..." });
                GetTagNames();
                progress.Report(new ProgressStatus() { Step = "Tag names found: " + string.Join(", ", TagNames.ToArray()) });
            }
            progress.Report(new ProgressStatus() { Step = "Calculating the number of reviews required for note selection..." });
            CalcNumberOfReviewsNeccesary(BaseNoteList.Notes.Count);
            progress.Report(new ProgressStatus() { Step = "Number of reviews required: " + _totalNrOfReviews });
            progress.Report(new ProgressStatus() { Step = "Generating review schedule..." });
            BuildMasterNoteList();
            Id = _reviewSetupService.Insert(this); //must insert here because we need a database id when generating the reviews.
            GenerateReviews();
            _reviewSetupService.Save(this); //call Update() to persist the already inserted ReviewSetup object with the generated reviews.
            progress.Report(new ProgressStatus() { Step = "Successfully completed filter setup.", IsComplete = true });
        });
    }
}

ReviewSetupApiModule.cs (the Nancy module receiving the request and callin DoJobAsync())

        Post["/newasync"] = parameters =>
        {
            this.RequiresAuthentication();
            try
            {

                //removed some stuff

                var progresss = new Progress<ProgressStatus>();
                var job = ProgressJobManager.Instance.DoJobAsync(j =>
                {                        
                    //EventHandler bound at the wrong moment/on the wrong thread?
                    progresss.ProgressChanged += delegate(object sender, ProgressStatus status)
                    {
                        j.ReportStep(status.Step);
                        if (status.IsComplete == true)
                        {
                            j.ReportComplete();
                        }
                    };   
                    setup.Setup(config, setupModel.NotebookGuid, setupModel.TagGuids, user, setupModel.ReviewName, progresss);
                });
                //this method needs to keep executing so it can return the job.Id, needed by the client to track its progress.
                return Response.AsJson(job).WithStatusCode(HttpStatusCode.OK);
            }
            catch (Exception ex)
            {
                return HttpStatusCode.BadRequest;
            }
        };

ProgressJob.cs

public class ProgressJob
{
    public event EventHandler<EventArgs> ProgressChanged;
    public event EventHandler<EventArgs> StepAdded;
    public event EventHandler<EventArgs> Completed;

    private volatile int _progress;
    private volatile bool _completed;
    private volatile List<string> _steps;
    private CancellationTokenSource _cancellationTokenSource;

    public List<string> Steps
    {
        get
        {
            return _steps;
        }
    }

    public ProgressJob(string id)
    {
        Id = id;
        _cancellationTokenSource = new CancellationTokenSource();
        _steps = new List<string>();
        _completed = false;
    }

    public string Id { get; private set; }

    public int Progress
    {
        get { return _progress; }
    }

    public bool IsComplete
    {
        get { return _completed; }
    }

    public CancellationToken CancellationToken
    {
        get { return _cancellationTokenSource.Token; }
    }

    public Progress<ProgressStatus> ProgressStatus { get; set; } 

    public void ReportProgress(int progress)
    {
        if (_progress != progress)
        {
            _progress = progress;
            OnProgressChanged();
        }
    }

    public void ReportStep(string step)
    {
        _steps.Add(step);
        OnStepAdded();
    }

    public void ReportComplete()
    {
        if (!IsComplete)
        {
            _completed = true;
            OnCompleted();
        }
    }

    protected virtual void OnCompleted()
    {
        var handler = Completed;
        if (handler != null) handler(this, EventArgs.Empty);
    }

    protected virtual void OnProgressChanged()
    {
        var handler = ProgressChanged;
        if (handler != null) handler(this, EventArgs.Empty);
    }

    protected virtual void OnStepAdded()
    {
        var handler = StepAdded;
        if (handler != null) handler(this, EventArgs.Empty);
    }

    public void Cancel()
    {
        _cancellationTokenSource.Cancel();
    }
}

ProgressJobManager.cs:

public class ProgressJobManager
{
    public static ProgressJobManager Instance;
    private IHubContext _hubContext;
    private IProgressJobDataProvider _jobDataProvider;

    public ProgressJobManager(IProgressJobDataProvider jobDataProvider)
    {
        _jobDataProvider = jobDataProvider;
        _hubContext = GlobalHost.ConnectionManager.GetHubContext<ProgressHub>();
    }

    public ProgressJob DoJobAsync(Action<ProgressJob> action)
    {
        var job = new ProgressJob(Guid.NewGuid().ToString());
        _jobDataProvider.AddJob(job);
        job.ProgressStatus = new Progress<ProgressStatus>();
        Task.Factory.StartNew(() =>
        {
            action(job);
            job.ReportComplete();
            job = _jobDataProvider.DeleteJob(job);
        },
        TaskCreationOptions.LongRunning);
        BroadcastJobStatus(job);
        return job;
    }

    private void BroadcastJobStatus(ProgressJob job)
    {
        job.ProgressChanged += HandleProgressChanged;
        job.Completed += HandleJobCompleted;
        job.StepAdded += HandleStepAdded;
    }

    private void HandleJobCompleted(object sender, EventArgs e)
    {
        var job = (ProgressJob)sender;

        _hubContext.Clients.Group(job.Id).JobCompleted(job.Id);

        job.ProgressChanged -= HandleProgressChanged;
        job.Completed -= HandleJobCompleted;
        job.StepAdded -= HandleStepAdded;
    }

    private void HandleProgressChanged(object sender, EventArgs e)
    {
        var job = (ProgressJob)sender;
        _hubContext.Clients.Group(job.Id).ProgressChanged(job.Id, job.Progress);
    }

    private void HandleStepAdded(object sender, EventArgs e)
    {
        var job = (ProgressJob)sender;
        var step = job.Steps[job.Steps.Count - 1];
        _hubContext.Clients.Group(job.Id).StepAdded(job.Id, step);
    }

    public ProgressJob GetJob(string id)
    {
        return _jobDataProvider.GetJob(id);
    }
}

ProgressHub.cs:

public class ProgressHub : Hub
{
    public void TrackJob(string jobId)
    {
        Groups.Add(Context.ConnectionId, jobId);       
    }

    public void CancelJob(string jobId)
    {
        var job = ProgressJobManager.Instance.GetJob(jobId);
        if (job != null)
        {
            job.Cancel();
        }
    }

    public void ProgressChanged(string jobId, int progress)
    {
        Clients.Group(jobId).ProgressChanged(jobId, progress);
    }

    public void JobCompleted(string jobId)
    {
        Clients.Group(jobId).JobCompleted(jobId);
    }

    public void StepAdded(string jobId, string step)
    {
        Clients.Group(jobId).StepAdded(jobId, step);
    }
}

And finally, ProgressStatus.cs:

public class ProgressStatus
{
    public bool IsComplete { get; set; }
    public int Percentage { get; set; }
    public string Step { get; set; }
}

Answer 1

A quick glance at the code suggests the problem is in ReviewSetupApiModule.cs where you call setup.Setup but don't await the task it returns. A secondary issue (probably a typo in the question) is that the method in ReviewSetup.cs is called SetupAsync.

When you don't await a task, the calling code will continue running before the task is finished. In this case, this will lead to the JobManager thinking the Job is finished, and removing it from the list.