MassTransit with RabbitMq Request/Response wrong reply address because of network segments

Question

I have a web app that uses a request/response message in Masstransit. This works on out test environment, no problem.

However on the customer deployment we face a problem. At the customer site we do have two network segments A and B. The component doing the database call is in segment A, the web app and the RabbitMq server in segment B.

Due to security restrictions the component in segment A has to go through a loadbalancer with a given address. The component itself can connect to RabbitMQ with Masstransit. So far so good.

The web component on segment B however uses the direct address for the RabbitMq server. When the web component now is starting the request/response call, I can see that the message arrives at the component in segment A. However I see that the consumer tries to call the RabbitMQ server on the "wrong" address. It uses the address the web component uses to issue the request. However the component in segment A should reply on the "loadbalancer" address.

Is there a way to configure or tell the RespondAsync call to use the connection address configured for that component?

Of course the easiest would be to have the web component also connect through the loadbalancer, but due to the network segments/security setup the loadbalancer is only reachable from segment A.

Any input/help is appreciated.

Answer 1

I had a similar problem with rabbitmq federation. Here's what I did.

ResponseAddressSendObserver

class ResponseAddressSendObserver : ISendObserver
{
    private readonly string _hostUriString;

    public ResponseAddressSendObserver(string hostUriString)
    {
        _hostUriString = hostUriString;
    }

    public Task PreSend<T>(SendContext<T> context)
        where T : class
    {
        if (context.ResponseAddress != null)
        {
            // Send relative response address alongside the message
            context.Headers.Set("RelativeResponseAddress",
                context.ResponseAddress.AbsoluteUri.Substring(_hostUriString.Length));
        }
        return Task.CompletedTask;
    }
    ...
}

ResponseAddressConsumeFilter

class ResponseAddressConsumeFilter : IFilter<ConsumeContext>
{
    private readonly string _hostUriString;

    public ResponseAddressConsumeFilter(string hostUriString)
    {
        _hostUriString = hostUriString;
    }

    public Task Send(ConsumeContext context, IPipe<ConsumeContext> next)
    {
        var responseAddressOverride = GetResponseAddress(_hostUriString, context);

        return next.Send(new ResponseAddressConsumeContext(responseAddressOverride, context));
    }

    public void Probe(ProbeContext context){}

    private static Uri GetResponseAddress(string host, ConsumeContext context)
    {
        if (context.ResponseAddress == null)
            return context.ResponseAddress;

        object relativeResponseAddress;
        if (!context.Headers.TryGetHeader("RelativeResponseAddress", out relativeResponseAddress) || !(relativeResponseAddress is string))
            throw new InvalidOperationException("Message has ResponseAddress but doen't have RelativeResponseAddress header");

        return new Uri(host + relativeResponseAddress);
    }
}

ResponseAddressConsumeContext

class ResponseAddressConsumeContext : BaseConsumeContext
{
    private readonly ConsumeContext _context;

    public ResponseAddressConsumeContext(Uri responseAddressOverride, ConsumeContext context)
        : base(context.ReceiveContext)
    {
        _context = context;
        ResponseAddress = responseAddressOverride;
    }

    public override Uri ResponseAddress { get; }

    public override bool TryGetMessage<T>(out ConsumeContext<T> consumeContext)
    {
        ConsumeContext<T> context;
        if (_context.TryGetMessage(out context))
        {
            // the most hackish part in the whole arrangement
            consumeContext = new MessageConsumeContext<T>(this, context.Message);
            return true;
        }
        else
        {
            consumeContext = null;
            return false;
        }
    }

    // all other members just delegate to _context 
}

And when configuring the bus

var result = MassTransit.Bus.Factory.CreateUsingRabbitMq(cfg =>
{
    var host = cfg.Host(new Uri(hostAddress), h =>
    {
        h.Username(...);
        h.Password(...);
    });

    cfg.UseFilter(new ResponseAddressConsumeFilter(hostAddress));

    ...
});
result.ConnectSendObserver(new ResponseAddressSendObserver(hostAddress));

So now relative response addresses are sent with the messages and used on the receiving side.

Using observers to modify anything is not recommended by the documentation, but should be fine in this case.

Maybe three is a better solution, but I haven't found one. HTH