Parallel.ForEach上的AccessViolationException

Question

Trying to run a Parallel.ForEach to lookup results from a external library, Z4DLL_NET. Documentation for the dll says the type is multi-thread safe. We have a large dataset which we are doing address verification on every month.

When running a any batch size larger than 1, I get the Access Violation Exception error on the _accumail.Lookup() in Lookup.

I tried to reduce the amount of threads by using the MaxDegreeOfParallelism but it did not prevent this issue. Any thoughts would be greatly appreciated.

Web Service Code:

    public void ProcessByBatchId(int batchId, int batchSize)
    {
        // get addresses to process
        var allAddresses = GetAddresses(batchId);

        var count = 0;

        // get initial set of addresses to process
        var addresses = ParseAddresses(allAddresses, count, batchSize).ToList();

        while (addresses.Any())
        {
            count += addresses.Count();

            // connect to db
            using (var entities = new Entities())
            {
                // turn these options off since they aren't needed here
                entities.Configuration.AutoDetectChangesEnabled = false;
                entities.Configuration.ValidateOnSaveEnabled = false;
                entities.Configuration.ProxyCreationEnabled = false;
                entities.Configuration.LazyLoadingEnabled = false;

                // process each address in parallel
                Parallel.ForEach(
                    addresses, 
                    addr =>
                {
                    // create dictionary for processing
                    var fields = GetFields(addr);

                    using (var addressValidator = _addressValidatorFactory.Create())
                    {
                        // lookup
                        var results = addressValidator.Lookup(fields);

                        SetResults(addr, results);
                    }
                });

                // set entity as changed for update
                addresses.ForEach(addr => entities.Entry(addr).State = EntityState.Modified);

                // commit changes to db
                entities.SaveChanges();

                // get next set of addresses to process
                addresses = ParseAddresses(allAddresses, count, batchSize).ToList();
            }
        }


    }

Lookup Code:

    public ValidationResults Lookup(IDictionary<FieldEnum, string> values)
    {
        IDictionary<FieldEnum, string> results = null;

        try
        {
            // load each value into accumail obj
            foreach (var field in Enum.GetNames(typeof(FieldEnum)))
            {
                var z4Field = (Z4DLL.Field)Enum.Parse(typeof(Z4DLL.Field), field);
                var fieldEnum = (FieldEnum)Enum.Parse(typeof(FieldEnum), field);

                if (values.ContainsKey(fieldEnum))
                {
                    _accumail.PutField(z4Field, values[fieldEnum] ?? string.Empty);
                    continue;
                }

                _accumail.PutField(z4Field, string.Empty);
            }

            // perform lookup
            if (_accumail.Lookup())
            {
                results = new Dictionary<FieldEnum, string>();

                // get each field from accumail obj
                foreach (var field in Enum.GetNames(typeof(FieldEnum)))
                {
                    results.Add((FieldEnum)Enum.Parse(typeof(FieldEnum), field),
                                _accumail.GetField((Z4DLL.Field)Enum.Parse(typeof(Z4DLL.Field), field)));
                }
            }

            var errorNum = _accumail.GetErrorNum();

            return new ValidationResults(results, errorNum, _accumail.GetErrorMsg(errorNum));
        }
        catch
        {
            var errorNum = _accumail.GetErrorNum();

            return new ValidationResults(results, errorNum, _accumail.GetErrorMsg(errorNum));
        }
    }

Error Description:

System.AccessViolationException was unhandled HResult=-2147467261
Message=Attempted to read or write protected memory. This is often an indication that other memory is corrupt. Source=Z4DLL32_NET
StackTrace: at Smartsoft.Toolkit.Z4DLL.Lookup() at Accumail.AccumailAddressValidator.Lookup(IDictionary 2 values) in AccumailAddressValidator.cs:line 50 at AddressValidationService.ProcessByBatchId>b__3_0(address_validation_detail addr) in AddressValidationService.svc.cs:line 145 at System.Threading.Tasks.Parallel.<>c__DisplayClass31_0 2.b__0(Int32 i) at System.Threading.Tasks.Parallel.<>c__DisplayClass17_0 1.b__1() at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask) at System.Threading.Tasks.Task.<>c__DisplayClass176_0.b__0(Object ) at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task.Execute() at System.Threading.Tasks.Task.ExecutionContextCallback(Object obj) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callba ck, Object state, Boolean preserveSyncCtx) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot) at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution) at System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem() at System.Threading.ThreadPoolWorkQueue.Dispatch() at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

Edit:

The type is

private readonly Z4DLL _accumail;

That is part of the Smartsoft.Toolkit. When the AddressValidator gets initialized, the constructor has

_accumail = new Z4DLL(databasePath);

Answer 1

You might have better luck using only one instance of your dll and calling it multiple times.

 using (var addressValidator = _addressValidatorFactory.Create())
 {               {

Parallel.ForEach(
                addresses, 
                addr =>
            {
                // create dictionary for processing
                var fields = GetFields(addr);

                                        // lookup
                    var results = addressValidator.Lookup(fields);

                    SetResults(addr, results);
                }
            });
 }

NOTE The above is intended as an example - I didn't test your logic to see if this will work without making other changes. The point is to show how it can be pulledoutside the for loop.