Visual Studio 2015 Unit Test Inconsistency

Question

I am receiving inconsistent unit test results after moving to VS2015 after being on VS2013. On VS2013, the unit tests were consistently passing. In VS2015, the unit tests fail if all are run at once or it is an automated Jenkins build. If the unit tests are individually run or all the failed tests are run at once, they pass.

This may be an EF6 issue. The error message I receive when all tests are run is...

Test Name:  GetQuoteService_NoExceptions
Test FullName:  MEA.SmartApp.Business.Factories.APV.UnitTests.QuoteFactoryTest.GetQuoteService_NoExceptions
Test Source:    C:\MetLife_Sourcecode\Common\DEV2\MEA.Smartapp.Common\Unit Tests\MEA.SmartApp.Business.Factories.APV.UnitTests\QuoteFactoryTest.cs : line 15
Test Outcome:   Failed
Test Duration:  0:00:00.00737

Result StackTrace:

at System.Data.Entity.Infrastructure.DependencyResolution.ProviderServicesFactory.GetInstance(String providerTypeName, String providerInvariantName)
   at System.Data.Entity.Internal.AppConfig.<.ctor>b__2(ProviderElement e)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at System.Data.Entity.Internal.AppConfig.<.ctor>b__1()
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.LazyInitValue()
   at System.Lazy`1.get_Value()
   at System.Data.Entity.Internal.AppConfig.get_DbProviderServices()
   at System.Data.Entity.Infrastructure.DependencyResolution.AppConfigDependencyResolver.RegisterDbProviderServices()
   at System.Data.Entity.Infrastructure.DependencyResolution.AppConfigDependencyResolver.GetServiceFactory(Type type, String name)
   at System.Data.Entity.Infrastructure.DependencyResolution.AppConfigDependencyResolver.<>c__DisplayClass1.<GetService>b__0(Tuple`2 t)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at System.Data.Entity.Infrastructure.DependencyResolution.AppConfigDependencyResolver.GetService(Type type, Object key)
   at System.Data.Entity.Infrastructure.DependencyResolution.DbDependencyResolverExtensions.GetServiceAsServices(IDbDependencyResolver resolver, Type type, Object key)
   at System.Data.Entity.Infrastructure.DependencyResolution.AppConfigDependencyResolver.GetServices(Type type, Object key)
   at System.Data.Entity.Infrastructure.DependencyResolution.ResolverChain.<>c__DisplayClass6.<GetServices>b__5(IDbDependencyResolver r)
   at System.Linq.Enumerable.<SelectManyIterator>d__1`2.MoveNext()
   at System.Linq.Enumerable.<ConcatIterator>d__1`1.MoveNext()
   at System.Linq.Enumerable.<OfTypeIterator>d__1`1.MoveNext()
   at System.Data.Entity.Utilities.IEnumerableExtensions.Each[T](IEnumerable`1 ts, Action`1 action)
   at System.Data.Entity.Infrastructure.DependencyResolution.InternalConfiguration.Lock()
   at System.Data.Entity.Infrastructure.DependencyResolution.DbConfigurationManager.<.ctor>b__1()
   at System.Lazy`1.CreateValue()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Lazy`1.get_Value()
   at System.Data.Entity.Infrastructure.DependencyResolution.DbConfigurationManager.GetConfiguration()
   at System.Data.Entity.DbContext.InitializeLazyInternalContext(IInternalConnection internalConnection, DbCompiledModel model)
   at System.Data.Entity.DbContext..ctor(String nameOrConnectionString)
   at MEA.SmartApp.Contexts.APV.AVMDB..ctor() in C:\MetLife_Sourcecode\Common\DEV2\MEA.Smartapp.Common\DataAccessLayer\MEA.SmartApp.Contexts.APV\AVMDB.Context.cs:line 19
   at MEA.SmartApp.Business.Factories.APV.QuoteFactory.GetAvmBusinessService() in C:\MetLife_Sourcecode\Common\DEV2\MEA.Smartapp.Common\BusinessLogicLayer\MEA.SmartApp.Business.Factories.APV\QuoteFactory.cs:line 65
   at MEA.SmartApp.Business.Factories.APV.QuoteFactory.GetQuoteService(APVTransactionCode transactionCode) in C:\MetLife_Sourcecode\Common\DEV2\MEA.Smartapp.Common\BusinessLogicLayer\MEA.SmartApp.Business.Factories.APV\QuoteFactory.cs:line 46
   at MEA.SmartApp.Business.Factories.APV.UnitTests.QuoteFactoryTest.GetQuoteService_NoExceptions() in C:\MetLife_Sourcecode\Common\DEV2\MEA.Smartapp.Common\Unit Tests\MEA.SmartApp.Business.Factories.APV.UnitTests\QuoteFactoryTest.cs:line 18
Result Message: 
Test method MEA.SmartApp.Business.Factories.APV.UnitTests.QuoteFactoryTest.GetQuoteService_NoExceptions threw exception: 
**System.InvalidOperationException: The Entity Framework provider type 'System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer' registered in the application config file for the ADO.NET provider with invariant name 'System.Data.SqlClient' could not be loaded**. Make sure that the assembly-qualified name is used and that the assembly is available to the running application. See http://go.microsoft.com/fwlink/?LinkId=260882 for more information.

*I realize this isnt a true unit test since it is trying to connect to a DB, so if you have any suggestions of how to fix that, let me know.

Answer 1

@KeithPayne - After running the failing unit tests with different sets of other unit tests, I was able to prove that it was one other test that caused the entity framework tests to fail. So you were right about the parallel execution.

There were two outstanding attributes of that unit test that caused the entity framework unit tests to fail.

1. The unit test covered VB.NET code while all others in my test suite tested C# code.
2. The unit test was initially created in VS2008 and had the test attribute of DeploymentItem("MIGTRX.dll"). MIGTRX is the VB.NET project that was under test. The reason for this attribute was to use the class under test's accessor class to publicly expose a private method so it could be unit tested. (Note: I realize this is a bad practice, but it was legacy code test that I inherited)