RBAC issues using MassTransit with Azure service bus and MSI authentication

Question

We are using MassTransit (v.5.5.5) with Azure Service Bus in a .net core v.2.2 application running in a container within Kubernetes. During the Bus setup we create a token using the managed service identity (MSI), like this...

var tokenProvider = TokenProvider.CreateManagedServiceIdentityTokenProvider();
var busControl = Bus.Factory.CreateUsingAzureServiceBus(cfg =>
{
    IServiceBusHost busHost = cfg.Host(new Uri(Settings.Host), h =>
    {
        h.OperationTimeout = TimeSpan.FromSeconds(5);
        h.TokenProvider = tokenProvider;
        h.TransportType = Settings.TransportType;
    });
    ...
}

Within Azure, the MSI is configured with all 3 Service Bus permissions (Owner, Sender and Receiver) - as decribed here .

When the application starts authentication seems to happen successfully but when MassTransit begins to listen on the specific queues we create it fails with the following error:

MassTransit.Azure.ServiceBus.Core.Transport.ReceiveTransport Error: 0 :
ReceiveTransport Faulted: sb://hostname.servicebus.windows.net/some-custom-queue, 
Microsoft.Azure.ServiceBus.UnauthorizedException: Generic: Failed during RBAC check, 
please make sure service resource provider is registered on your tenant.

Points to note:

• The application does create the temporary bus queue, and there are no errors to do with accessing it.
• The application does not create our custom queues.
• If the MSI permissions are removed then no queues are created and the error is different: Authorization failed for specified action: Manage,EntityWrite.
• For what it's worth we are using the AmqpWebSockets transport type.

It seems weird that the temporary queue is created but not our application-specific ones.

So I guess my question is this... have I done anything wrong or is this something that is not yet available for MassTransit using Service Bus with MSI?

Thanks in advance.

Answer 1

We came across the same issue. We are using Mass Transit 6.* and Azure Service Bus. When switched to MSI we start see the same error:

Microsoft.Azure.ServiceBus.UnauthorizedException: Generic: Failed during RBAC check, please make sure service resource provider is registered on your tenant. TrackingId:2644d62a-83a9-444f-8aeb-b4bdee2fa84d_G30, SystemTracker:NoSystemTracker, Timestamp:2020-02-13T15:54:49
    at Microsoft.Azure.ServiceBus.Management.ManagementClient.SendHttpRequest(HttpRequestMessage request, CancellationToken cancellationToken)
    at Microsoft.Azure.ServiceBus.Management.ManagementClient.PutEntity(String path, String requestBody, Boolean isUpdate, String forwardTo, String fwdDeadLetterTo, CancellationToken cancellationToken)
    at Microsoft.Azure.ServiceBus.Management.ManagementClient.CreateTopicAsync(TopicDescription topicDescription, CancellationToken cancellationToken)
    at MassTransit.Azure.ServiceBus.Core.Contexts.NamespaceManager.<>c__DisplayClass25_0`1.<<RunOperation>b__0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
    at Microsoft.Azure.ServiceBus.RetryPolicy.RunOperation(Func`1 operation, TimeSpan operationTimeout)
    at Microsoft.Azure.ServiceBus.RetryPolicy.RunOperation(Func`1 operation, TimeSpan operationTimeout)
    at MassTransit.Azure.ServiceBus.Core.Contexts.NamespaceManager.RunOperation[T](Func`1 operation)
    at MassTransit.Azure.ServiceBus.Core.Contexts.ServiceBusNamespaceContext.CreateTopic(TopicDescription topicDescription)
    at MassTransit.Azure.ServiceBus.Core.Pipeline.ConfigureTopologyFilter`1.ConfigureTopology(NamespaceContext context)
    at MassTransit.Azure.ServiceBus.Core.Pipeline.ConfigureTopologyFilter`1.<>c__DisplayClass5_0.<<Send>b__0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
    at GreenPipes.PipeExtensions.OneTimeSetup[T](PipeContext context, Func`2 setupMethod, PayloadFactory`1 payloadFactory)
    at MassTransit.Azure.ServiceBus.Core.Pipeline.ConfigureTopologyFilter`1.Send(NamespaceContext context, IPipe`1 next)
    at GreenPipes.Agents.AsyncPipeContextPipe`1.Send(TContext context)
    at GreenPipes.Agents.PipeContextSupervisor`1.GreenPipes.IPipeContextSource<TContext>.Send(IPipe`1 pipe, CancellationToken cancellationToken)
    at GreenPipes.Agents.PipeContextSupervisor`1.GreenPipes.IPipeContextSource<TContext>.Send(IPipe`1 pipe, CancellationToken cancellationToken)
    at GreenPipes.Agents.PipeContextSupervisor`1.GreenPipes.IPipeContextSource<TContext>.Send(IPipe`1 pipe, CancellationToken cancellationToken)
    at MassTransit.Azure.ServiceBus.Core.Pipeline.JoinContextFactory`3.<>c__DisplayClass7_0.<<CreateJoinContext>g__Join|0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
    at MassTransit.Azure.ServiceBus.Core.Pipeline.ClientContextFactory.CreateSharedContext(Task`1 context, CancellationToken cancellationToken)
    at GreenPipes.Agents.PipeContextSupervisor`1.GreenPipes.IPipeContextSource<TContext>.Send(IPipe`1 pipe, CancellationToken cancellationToken)
    at GreenPipes.Agents.PipeContextSupervisor`1.GreenPipes.IPipeContextSource<TContext>.Send(IPipe`1 pipe, CancellationToken cancellationToken)
    at GreenPipes.Agents.PipeContextSupervisor`1.GreenPipes.IPipeContextSource<TContext>.Send(IPipe`1 pipe, CancellationToken cancellationToken)
    at MassTransit.Azure.ServiceBus.Core.Transport.ReceiveTransport.<Receiver>b__12_0()
    at MassTransit.Azure.ServiceBus.Core.Pipeline.JoinContextFactory`3.<>c__DisplayClass7_0.<<CreateJoinContext>g__Join|0>d.MoveNext()

Finally we found that ASB Sdk Microsoft.Azure.ServiceBus.Management.ManagementClient doesn't allow to create topics with slashes in path when tokens from MSI used. This code below return the same error:

         try
            {
                var address = "sb://[name].servicebus.windows.net";
                var tokenProvider = new TokenProvider.CreateManagedIdentityTokenProvider();();
                var _managementClient = new ManagementClient(address.ToString(), tokenProvider);
                var topicDefinition = new TopicDescription("myNewTopic/22222");
                await _managementClient.CreateTopicAsync(topicDefinition);

            }
            catch (Exception e)
            {
                Debug.WriteLine(e);

                throw;
            }

Since MT uses special naming convention to setup messaging topology, we get topic path with namespace of particular message type. If we have message MassTransitTest.Message1 it will be converted to topic MassTransitTest/Message1 (with slash). To fix this we setup our own IEntityNameFormatter

public class MyEntityNameFormatter : IEntityNameFormatter
{
    readonly IMessageNameFormatter _formatter;

    public MyEntityNameFormatter()
    {
        _formatter = new ServiceBusMessageNameFormatter();
    }
    public string FormatEntityName<T>()
    {
        var entityName = _formatter.GetMessageName(typeof(T)).ToString().Replace("/",".");
        return entityName;
    }
}

and then

 var bus = Bus.Factory.CreateUsingAzureServiceBus(configurator =>
            {
                ...
                configurator.MessageTopology.SetEntityNameFormatter(new MyEntityNameFormatter());
                ...

This fixed our issue. (We tried use ~ instead of. but got some validation errors, so decided use dots)

Answer 2

I have been able to reproduce the same error using Azure Service Bus SDK code directly in a .NET Core (3.0) ASP.NET application hosted in an AppService with Managed Identity.

string serviceBusNamespace = "busnamespace_here";

string queueName = "with_service_name/test"; // NOT working with RBAC
// string queueName = "testrbac"; // Works with RBAC - no /

var msiTokenProvider = TokenProvider.CreateManagedIdentityTokenProvider();
var qc = new QueueClient(new Uri($"sb://{serviceBusNamespace}.servicebus.windows.net/").ToString(), queueName, msiTokenProvider);

var message = new Message(Encoding.UTF8.GetBytes($"Test {DateTime.Now.ToShortTimeString()}"))
{
    ContentType = "application/text",
    Label = "Scientist",
    MessageId = Guid.NewGuid().ToString()
};

try
{
    await qc.SendAsync(message);
}
catch (Exception ex)
{
    return View(ex.ToString());
}

I found out that the error seems to be caused by the presence of a / in the queue name. It's a character that is allowed in the Service Bus but it doesn't seems to works quite nicely with RBAC Service Bus feature.
The same code works well if you use a SAS Policy instead of Managed Identity/Client Secret.

Here is my test Service Bus in image (using Service Bus Explorer):

Mass Transit

I know Mass Transit can use entity path with slash character in it. Does that means we can't use RBAC to access those entities?

I wasn't able to find what is the root cause.