通過基於任務的異步模式 (TAP) 接觸的 Windows 服務作業監視器

Question

我正在用 C# 構建一個 Windows 服務，它將監視一個作業隊列，當它在隊列中找到可用的項目時，它會啟動將“完全”處理該作業（包括失敗）的作業。 我正在使用 Task.Factory.StartNew() 並且無可否認，我對 TAP 非常熟悉（在完成這篇文章后開始閱讀博客）。

要求

1. 定期輪詢數據庫隊列以查找可用作業。 （為了這個問題，讓我們忽略消息隊列與數據庫隊列的論點）

2. 異步啟動作業，以便輪詢可以繼續監視隊列並啟動新作業。

3. 遵守“工作”門檻，以免產生過多工作。

4. 如果作業正在處理，則“延遲”關閉服務。

5. 確保將作業中的失敗記錄到事件日志中，並且不會使 Windows 服務崩潰。

我的代碼在下面，但這里是我的問題/疑慮（如果有更好的地方可以發布此內容，請告訴我）但這主要圍繞 TAP 的正確使用和它的“穩定性”。 請注意，我的大部分問題也記錄在代碼中。

問題

1. 在 PollJobs 中，我使用 Task.Factory.Start/ContinueWith 的方式是否正確使用此作業處理服務來保持高吞吐量？ 我永遠不會阻塞任何線程，並希望對我目前擁有的一點點 TAP 使用正確的模式。

2. ConcurrentDictionary - 使用它來監視當前正在運行的作業，並且每個作業在完成時都會從字典中刪除自己（在我從 Task.Factory.StartNew 假設的單獨線程上），因為它是 ConcurrentDictionary，我假設我不需要任何使用時隨處鎖定？

3. 作業異常（最糟糕的是 OutOfMemoryException） - 作業處理期間的任何異常都無法關閉服務，必須正確記錄在事件日志和數據庫隊列中。 目前，不幸的是，有些作業會拋出 OutOfMemoryException。 “作業處理”中的 try/catch 是否足以捕獲和處理所有場景，以便 Windows 服務永遠不會意外終止？ 或者為每個工作啟動一個 AppDomain 以獲得更多隔離會更好/更安全嗎？ （過殺？）

4. 我見過爭論使用“正確”計時器的爭論，但沒有像樣的答案。 對我的 System.Threading.Timer 的設置和使用有什么意見嗎？ （特別是關於我如何確保在上一次調用完成之前永遠不會再次調用 PollJobs）

如果你已經做到了這一步。 提前致謝。

代碼

public partial class EvolutionService : ServiceBase
{
    EventLog eventLog = new EventLog() { 
        Source = "BTREvolution", 
        Log = "Application" 
    };
    Timer timer;
    int pollingSeconds = 1;

    // Dictionary of currently running jobs so I can query current workload.  
    // Since ConcurrentDictionary, hoping I don't need any lock() { } code.
    private ConcurrentDictionary<Guid, RunningJob> runningJobs = 
        new ConcurrentDictionary<Guid, RunningJob>();

    public EvolutionService( string[] args )
    {
        InitializeComponent();

        if ( !EventLog.SourceExists( eventLog.Source ) )
        {
            EventLog.CreateEventSource( 
                eventLog.Source, 
                eventLog.Log );
        }
    }

    protected override void OnStart( string[] args )
    {
        // Only run polling code one time and the PollJobs will 
        // initiate next poll interval so that PollJobs is never 
        // called again before it finishes its processing, http://stackoverflow.com/a/1698409/166231
        timer = new System.Threading.Timer( 
            PollJobs, null, 
            TimeSpan.FromSeconds( 5 ).Milliseconds, 
            Timeout.Infinite );
    }

    protected override void OnPause()
    {
        // Disable the timer polling so no more jobs are processed
        timer = null;

        // Don't allow pause if any jobs are running
        if ( runningJobs.Count > 0 )
        {
            var searcher = new System.Management.ManagementObjectSearcher( 
                "SELECT UserName FROM Win32_ComputerSystem" );
            var collection = searcher.Get();
            var username = 
                (string)collection
                    .Cast<System.Management.ManagementBaseObject>()
                    .First()[ "UserName" ];
            throw new InvalidOperationException( $"{username} requested pause.  The service will not process incoming jobs, but it must finish the {runningJobs.Count} job(s) are running before it can be paused." );
        }

        base.OnPause();
    }

    protected override void OnContinue()
    {
        // Tell time to start polling one time in 5 seconds
        timer = new System.Threading.Timer( 
            PollJobs, null, 
            TimeSpan.FromSeconds( 5 ).Milliseconds, 
            Timeout.Infinite );
        base.OnContinue();
    }

    protected override void OnStop()
    {
        // Disable the timer polling so no more jobs are processed
        timer = null;

        // Until all jobs successfully cancel, keep requesting more time
        // http://stackoverflow.com/a/13952149/166231
        var task = Task.Run( () =>
        {
            // If any running jobs, send the Cancel notification
            if ( runningJobs.Count > 0 )
            {
                foreach ( var job in runningJobs )
                {
                    eventLog.WriteEntry( 
                        $"Cancelling job {job.Value.Key}" );
                    job.Value.CancellationSource.Cancel();
                }
            }

            // When a job cancels (and thus completes) it'll 
            // be removed from the runningJobs workload monitor.  
            // While any jobs are running, just wait another second
            while ( runningJobs.Count > 0 )
            {
                Task.Delay( TimeSpan.FromSeconds( 1 ) ).Wait();
            }
        } );

        // While the task is not finished, every 5 seconds 
        // I'll request an additional 5 seconds
        while ( !task.Wait( TimeSpan.FromSeconds( 5 ) ) )
        {
            RequestAdditionalTime( 
                TimeSpan.FromSeconds( 5 ).Milliseconds );
        }
    }

    public void PollJobs( object state )
    {
        // If no jobs processed, then poll at regular interval
        var timerDue = 
            TimeSpan.FromSeconds( pollingSeconds ).Milliseconds;

        try
        {
            // Could define some sort of threshhold here so it 
            // doesn't get too bogged down, might have to check
            // Jobs by type to know whether 'batch' or 'single' 
            // type jobs, for now, just not allowing more than
            // 10 jobs to run at once.
            var availableProcesses = 
                Math.Max( 0, 10 - runningJobs.Count );

            if ( availableProcesses == 0 ) return;

            var availableJobs = 
                JobProvider.TakeNextAvailableJobs( availableProcesses );
            foreach ( var j in availableJobs )
            {
                // If any jobs processed, poll immediately when finished
                timerDue = 0;

                var job = new RunningJob
                {
                    Key = j.jKey,
                    InputPackage = j.JobData.jdInputPackage,
                    DateStart = j.jDateStart.Value,
                    CancellationSource = new CancellationTokenSource()
                };

                // Add running job to the workload monitor
                runningJobs.AddOrUpdate(
                    j.jKey,
                    job,
                    ( key, info ) => null );

                Task.Factory
                    .StartNew(
                        i =>
                        {
                            var t = (Tuple<Guid, CancellationToken>)i;

                            var key = t.Item1; // Job Key
                            // Running process would check if cancel has been requested
                            var token = t.Item2; 
                            var totalProfilesProcess = 1;

                            try
                            {
                                eventLog.WriteEntry( $"Running job {key}" );

                                // All code in here completes the jobs.
                                // Will be a seperate method per JobType.
                                // Any exceptions in here *MUST NOT* 
                                // take down service.  Before allowing 
                                // the exception to propogate back up 
                                // into *this* try/catch, the code must 
                                // successfully clean up any resources 
                                // and state that was being modified so 
                                // that the client who submitted this 
                                // job is properly notified.

                                // This is just simulation of running a 
                                // job...so each job takes 10 seconds.
                                for ( var d = 0; d < 10; d++ )
                                {
                                    // or could do await if I called Unwrap(),
                                    // https://blogs.msdn.microsoft.com/pfxteam/2011/10/24/task-run-vs-task-factory-startnew/
                                    Task.Delay( 1000 ).Wait();
                                    totalProfilesProcess++;

                                    if ( token.IsCancellationRequested )
                                    {
                                        // TODO: Clean up the job
                                        throw new OperationCanceledException( token );
                                    }
                                }

                                // Success
                                JobProvider.UpdateJobStatus( key, 2, totalProfilesProcess );
                            }
                            catch ( OperationCanceledException )
                            {
                                // Cancelled
                                JobProvider.UpdateJobStatus( key, 3, totalProfilesProcess );
                                throw;
                            }
                            catch ( Exception )
                            {
                                // Failed
                                JobProvider.UpdateJobStatus( key, 4, totalProfilesProcess );
                                throw;
                            }
                        },
                        // Pass cancellation token to job so it can watch cancel request
                        Tuple.Create( j.jKey, job.CancellationSource.Token ),
                        // associate cancellation token with Task started via StartNew()
                        job.CancellationSource.Token, 
                        TaskCreationOptions.LongRunning,
                        TaskScheduler.Default

                    ).ContinueWith(
                        ( t, k ) =>
                        {
                            // When Job is finished, log exception if present.
                            // Haven't tested this yet, but think 
                            // Exception will always be AggregateException
                            // so I'll have to examine the InnerException.
                            if ( !t.IsCanceled && t.IsFaulted )
                            {
                                eventLog.WriteEntry( $"Exception for {k}: {t.Exception.Message}", EventLogEntryType.Error );
                            }

                            eventLog.WriteEntry( $"Completed job {k}" );

                            // Remove running job from the workload monitor
                            RunningJob completedJob;
                            runningJobs.TryRemove( 
                                (Guid)k, out completedJob );
                        },
                        j.jKey
                    );
            }
        }
        catch ( Exception ex )
        {
            // If can't even launch job, disable the polling.
            // TODO: Could have an error threshhold where I don't
            // shut down until X errors happens
            eventLog.WriteEntry( 
                ex.Message + "\r\n\r\n" + ex.StackTrace, 
                EventLogEntryType.Error );
            timer = null;
        }
        finally
        {
            // If timer wasn't 'terminated' in OnPause or OnStop, 
            // then set to call timer again
            if ( timer != null )
            {
                timer.Change( timerDue, Timeout.Infinite );
            }
        }
    }
}

class RunningJob
{
    public Guid Key { get; set; }
    public DateTime DateStart { get; set; }
    public XElement InputPackage { get; set; }
    public CancellationTokenSource CancellationSource { get; set; }
}

Answer 1

我用 Hangfire.io 解決了我的問題。