Troubleshoot Service Fabric service deployment

Question

We have a ServiceFabric service, which is running the latest .NET Core 3.1. We publish & deploy it as self-contained x64 app. After a recent update, we are unable to deploy it anymore. ServiceFabric gives very little information why. This is what we see:

Service package for manifest 'XXXX' and service package activation ID 'XXXX' is in Error.
        'System.Hosting' reported Error for property 'ServiceTypeRegistration:XXX'.
        The ServiceType was disabled on the node.

It tries that several times and then gives up. By digging into SF events I was able to find this as well:

There was an error during CodePackage activation.The process/container terminated with exit code:2147516556.

When using a local Service Fabric Cluster, everything is deployed just fine.

Does anybody know who to get more info out of the deployment process? More logs. Some clue which will help us what exactly is going wrong with the deployment.

Answer 1

So, indeed our code was not running at all. Thus, we did not get any logs from it. But, there is a way to get some log from Service Fabric runtime itself. As described here: https://devblogs.microsoft.com/premier-developer/service-fabric-how-to-troubleshoot-deployment-errors-in-service-fabric-windows-cluster/#:~:text=Debug%20service%20errors%20using%20the%20log%20files we added

<ConsoleRedirection FileRetentionCount="5" FileMaxSizeInKb="2048"/>

to our ServiceManifest.xml . As a result, we got some useful logs. Particularly we got this:

Error:
  An assembly specified in the application dependencies manifest (XXXX.deps.json) was not found:
    package: 'Microsoft.Data.SqlClient.SNI.runtime', version: '2.0.0'
    path: 'runtimes/win-x64/native/Microsoft.Data.SqlClient.SNI.pdb'

and this gives us some clues to start working on the actual problem.