Occasional error “Unable to save config to file '…\web.config'” on TeamCity

Question

We're occasionally getting this error on our Continuous Integration server (TeamCity):

Unable to save config to file '...\web.config'.

This happens in less than 10% of our builds and on different build agents, so I suppose it's not an issue with permissions.

Tests that fail call a WCF service that writes to web.config file of another service. It's NOT THREAD-SAFE , but as far as I know TeamCity doesn't run tests in parallel, otherwise a lot of other our tests would fail all the time.

I tried to reproduce the error by spawning a lot of threads that call configuration service and got another exception:

The process cannot access the file '...\web.config' because it is being used 
by another process

Can anyone help me to find the cause of the problem, so I can at least reproduce it with a stable test?

Answer 1

File locks on web.config aren't being released in time. When test A causes a write to web.config, it is not guaranteed that the file lock on web.config will be released by the time the test is done. The lock may still be there when test B runs and tries to write to web.config. It's not likely to happen much because locks are usually released quickly. However it does happen once in a while and that may account for your 10% failure rate.

Use a try catch block when writing to web.config and retry the write when an exception occurs. That's not 100% reliable so you may want to consider custom file locking.