通过 MassTransit 发布者/消费者传播 W3C 跟踪上下文

Question

'm trying to support propagating the W3C trace context traceId and spanId properties from http calls -> publisher -> consumer -> http call within MassTransit (just so they show up in the logs/seq for now, but we're using Dynatrace), but am I couldn't find anything out-of-the-box here: https://masstransit-project.com/advanced/monitoring/diagnostic-source.html

If there is nothing available I'll probably try to create something myself based on these articles:

I could find this one as an example for OpenTracing: https://github.com/yesmarket/MassTransit.OpenTracing

And this as a reference for NServiceBus: https://jimmybogard.com/building-end-to-end-diagnostics-and-tracing-a-primer-trace-context/

Unless anyone can suggest something that already exists?

Answer 1

EDIT2: The latest versions of MassTransit propagate the trace context by default.

Enable W3C Tracing in your startup:

Activity.DefaultIdFormat = ActivityIdFormat.W3C;

on configuring your bus call:

bus.UseInstrumentation();

EDIT: Based on the library by Ryan Bartsch & article by Jimmy Bogard, I created this package which does what I need it to do: https://github.com/zidad/MassTransit.ActivityTracing

Answer 2

Dynatracing claims to integrate seamlessly with OpenTracing https://www.dynatrace.com/integrations/opentracing/ and if you use the library you mentioned and if you have your HTTP part instrumented with OpenTracing, it will work out of the box.

The only potential drawback that in the service that hosts gets an HTTP call and within the context of handling it sends or publishes a message via MassTransit, everything must be instrumented with OpenTracing, because it will start the child span using the OpenTracing API.

We do this with Datadog, so we use the Datadog OpenTracing integration library and trace the WebApi and HttpClient using OpenTracing Contrib libraries. So, the automatic instrumentation didn't work for us. But it's not hard to use those libs to instrument your app instead of using the automatic instrumentation.

The usual flow is like:

• Outside -> WebApi: start a span
• WebApi -> MassTransit: start a child span, inject the context to headers
• MassTransit -> consumer: extract the context, start a child span and so on

Both bits with injection and extraction are handled in the MassTransit.OpenTracing library, so, there's nothing extra to do.

With a library that your provider has to support OpenTracing it's usually like this:

• Configure the tracer of the provider
• Set the OpenTracing global tracer to the wrapper, using the integration library from your provider
• When you create a span with OpenTracing, it will create a vendor-specific span and wrap it in OpenTracing span

Answer 3

I'm the author of the MassTransit.OpenTracing library you referenced, but I wrote it prior to the W3C recommendation, which looks to be quite recent (Feb 6, 2020).

My goal was that I wanted distributed tracing through a microservice architecture that had both synchronous/HTTP as well as asynchronous message broker communications. For the HTTP stuff I was using OpenTracing.Contrib.NetCore , which 'uses .NET's DiagnosticSource to instrument its code'. For the asynchronous message broker communications, I was using RabbitMQ with MassTransit, but I didn't really understand the MassTransit DiagnosticSource approach suggested on the website (and nor could I find any examples), so I decided to get into the nuts and bolts a bit and roll my own.

Long story short, it all worked as expected using Jaeger as the tracer. Interestingly, we (as in the company I work for) decided to also use DynaTrace, which operates at a much lower level and kind of removes the need for handling a lot of this stuff in code. That said, the approach is not invalid (IMO), as not everyone can afford DynaTrace (or similar APM tools).

I'll try to upgrade this library with the W3C recommendation in the coming week or 2. Let me know if you want to help with contribution/review (or if you want to go off in a different direction and roll your own is also fine)...