IdentityServer4 Using Client Credential Workflow with an API (Or trying to emulate OIDC calls)

Question

I am wanting to use IdentityServer4 to secure APIs using Windows Credentials. I have created a working example in a web application, but trying to mimic the OIDC calls is proving troublesome. In the documents it appears to suggest that the only way to work with an API is to authenticate with a ClientID and secret. I wanted to see if this was true. Below I will add my network calls I am currently doing to try and emulate the OIDC workflow. Hopefully there is either a better way to approach this problem or a simpler set of calls. I appreciate help either way.

Minimal working example (All calls made through Postman)

1. I call the login endpoint "[GET] https://localhost:44353/Account/Login", this returns a 200 OK login page HTML and more importantly my ".AspNetCore.Antiforgery" cookie

2. I call my challenge endpoint "[GET] https://localhost:44353/External/Challenge?provider=Windows" using NTLM Authentication and providing my windows credentials. This returns a 401 Unauthorized and a cookie "idsrv.external", I think the 401 is just due to a redirect, I actually just need the cookie.

3. I call the callback endpoint "[GET] https://localhost:44353/External/Callback" and that deletes my "idsrv.external" cookie and sets cookies called "idsrv.session" and "idsrv".

4. I now try and call my API endpoint "[GET] https://localhost:16385/managementservice/schema" using the cookies I have received so far. This returns to me the OIDC permissions request page.

5. I take the return URL and token from the html of the last request and I call "[POST] https://localhost:44353/Consent" with the form data below. This returns 200 OK html with a button that calls "https://localhost:16385/signin-oidc".

6. I use the data from the last html to then call "[POST] https://localhost:16385/signin-oidc" as shown below. I have 5 cookies set; .AspNetCore.OpenIdConnect.Nonce, .AspNetCore.Correlation.oidc, .AspNetCore.Antiforgery, idsrv.session, and idsrv. However, this call returns a 500 Internal Server Error instead of a 302 Found like the UI. Any ideas on what is going wrong here?

I can provide more data or specific files as needed. This is just a jumping off point.

EDIT: I received a request to provide applicable files. My client application is a ASP.NET Core API that I am htting with postman.

IdentityServer Startup.cs

using IdentityModel;
using IdentityServer4;
using IdentityServer4.Quickstart.UI;
using IdentityServer4.Services;
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Authentication.Cookies;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.IdentityModel.Tokens;
using System.IdentityModel.Tokens.Jwt;

namespace IdentityServerTemplate
{
    public class Startup
    {
        public IWebHostEnvironment Environment { get; }
        public IConfiguration Configuration { get; }

        public Startup(IWebHostEnvironment environment, IConfiguration configuration)
        {
            Environment = environment;
            Configuration = configuration;
        }

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddControllersWithViews();

            services.AddHttpClient();

            // configures IIS out-of-proc settings (see https://github.com/aspnet/AspNetCore/issues/14882)
            services.Configure<IISOptions>(iis =>
            {
                iis.AuthenticationDisplayName = "Windows";
                iis.AutomaticAuthentication = true;
            });

            // configures IIS in-proc settings
            services.Configure<IISServerOptions>(iis =>
            {
                iis.AuthenticationDisplayName = "Windows";
                iis.AutomaticAuthentication = true;
            });

            var builder = services.AddIdentityServer(options =>
            {
                options.Events.RaiseErrorEvents = true;
                options.Events.RaiseInformationEvents = true;
                options.Events.RaiseFailureEvents = true;
                options.Events.RaiseSuccessEvents = true;
            });
            //.AddTestUsers(TestUsers.Users);

            // in-memory, code config
            builder.AddInMemoryIdentityResources(Config.Ids);
            builder.AddInMemoryApiResources(Config.Apis);
            builder.AddInMemoryClients(Config.Clients);

            services.AddScoped<IProfileService, ADProfileService>();

            // or in-memory, json config
            //builder.AddInMemoryIdentityResources(Configuration.GetSection("IdentityResources"));
            //builder.AddInMemoryApiResources(Configuration.GetSection("ApiResources"));
            //builder.AddInMemoryClients(Configuration.GetSection("clients"));

            // not recommended for production - you need to store your key material somewhere secure
            builder.AddDeveloperSigningCredential();

            services.AddAuthentication();
                //.AddGoogle(options =>
                //{
                //    options.SignInScheme = IdentityServerConstants.ExternalCookieAuthenticationScheme;

                //    // register your IdentityServer with Google at https://console.developers.google.com
                //    // enable the Google+ API
                //    // set the redirect URI to http://localhost:5000/signin-google
                //    options.ClientId = "copy client ID from Google here";
                //    options.ClientSecret = "copy client secret from Google here";
                //});
        }

        public void Configure(IApplicationBuilder app)
        {
            if (Environment.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }

            app.UseStaticFiles();

            app.UseRouting();
            app.UseIdentityServer();
            app.UseAuthorization();
            app.UseAuthentication();
            app.UseEndpoints(endpoints =>
            {
                endpoints.MapDefaultControllerRoute();
            });
        }
    }
}

IdentityServer Config.cs

using IdentityModel;
using IdentityServer4.Models;
using System.Collections.Generic;

namespace IdentityServerTemplate
{
    public static class Config
    {
        public static IEnumerable<IdentityResource> Ids =>
            new IdentityResource[]
            {
                new IdentityResources.OpenId(),
                new IdentityResources.Profile(),
                new IdentityResources.Email(),
                new IdentityResources.Address(),
            };


        public static IEnumerable<ApiResource> Apis =>
            new ApiResource[]
            {
                // new ApiResource("api1", "My API #1")

                new ApiResource("api1", "My API", new[] { JwtClaimTypes.Subject, JwtClaimTypes.Email, JwtClaimTypes.Address, "upn_custom"})
            };


        public static IEnumerable<Client> Clients =>
            new Client[]
            {
                // client credentials flow client
                new Client
                {
                    ClientId = "identity.server",
                    ClientName = "Identity Server Client",

                    AllowedGrantTypes = GrantTypes.ClientCredentials,
                    AlwaysIncludeUserClaimsInIdToken = true,
                    ClientSecrets = { new Secret("secret".Sha256()) },

                    AllowedScopes = { "openid", "profile", "email", "address", "api1", "upn_custom" }
                },

                // MVC client using code flow + pkce
                new Client
                {
                    //ClientId = "mvc",
                    ClientId = "mvc.code",
                    ClientName = "MVC Client",

                    // Note
                    AlwaysIncludeUserClaimsInIdToken = true,

                    AllowedGrantTypes = GrantTypes.CodeAndClientCredentials,
                    //RequirePkce = true,
                    RequirePkce = false,
                    //ClientSecrets = { new Secret("49C1A7E1-0C79-4A89-A3D6-A37998FB86B0".Sha256()) },
                    ClientSecrets = { new Secret("secret".Sha256()) },

                    //RedirectUris = { "https://localhost:5003/signin-oidc" },
                    RedirectUris = { "https://localhost:5003/signin-oidc" },
                    FrontChannelLogoutUri = "https://localhost:5003/signout-oidc",
                    PostLogoutRedirectUris = { "https://localhost:5003/signout-callback-oidc" },

                    AllowOfflineAccess = true,
                    AllowedScopes = { "openid", "profile", "email", "address", "api1", "upn_custom" }
                },

                // MCW Appserver
                new Client
                {
                    //ClientId = "mvc",
                    ClientId = "mcw.appserver",
                    ClientName = "MCW AppServer",

                    // Note
                    AlwaysIncludeUserClaimsInIdToken = true,

                    AllowedGrantTypes = GrantTypes.CodeAndClientCredentials,
                    RequirePkce = false,
                    //RequirePkce = false,
                    //ClientSecrets = { new Secret("49C1A7E1-0C79-4A89-A3D6-A37998FB86B0".Sha256()) },
                    ClientSecrets = { new Secret("secret".Sha256()) },

                    //RedirectUris = { "http://localhost:16835/signin-oidc" },
                    RedirectUris = { "https://localhost:16385/signin-oidc" },
                    FrontChannelLogoutUri = "https://localhost:16835/signout-oidc",
                    PostLogoutRedirectUris = { "https://localhost:16835/signout-callback-oidc" },

                    AllowOfflineAccess = true,
                    AllowedScopes = { "openid", "profile", "email", "address", "api1", "upn_custom" }
                },

                // MVC client using code flow + pkce
                new Client
                {
                    //ClientId = "mvc",
                    ClientId = "ptp.appserv",
                    ClientName = "PTP AppServ",

                    // Note
                    AlwaysIncludeUserClaimsInIdToken = true,

                    AllowedGrantTypes = GrantTypes.CodeAndClientCredentials,
                    //RequirePkce = true,
                    RequirePkce = false,
                    //ClientSecrets = { new Secret("49C1A7E1-0C79-4A89-A3D6-A37998FB86B0".Sha256()) },
                    ClientSecrets = { new Secret("secret".Sha256()) },

                    //RedirectUris = { "https://localhost:30001/signin-oidc" },
                    RedirectUris = { "https://localhost:30001/signin-oidc" },
                    FrontChannelLogoutUri = "https://localhost:30001/signout-oidc",
                    PostLogoutRedirectUris = { "https://localhost:30001/signout-callback-oidc" },

                    AllowOfflineAccess = true,
                    AllowedScopes = { "openid", "profile", "email", "address", "api1", "upn_custom" }
                },             

                // SPA client using code flow + pkce
                new Client
                {
                    ClientId = "spa",
                    ClientName = "SPA Client",
                    ClientUri = "http://identityserver.io",

                    AllowedGrantTypes = GrantTypes.Code,
                    RequirePkce = true,
                    RequireClientSecret = false,

                    RedirectUris =
                    {
                        "http://localhost:5002/index.html",
                        "http://localhost:5002/callback.html",
                        "http://localhost:5002/silent.html",
                        "http://localhost:5002/popup.html",
                    },

                    PostLogoutRedirectUris = { "http://localhost:5002/index.html" },
                    AllowedCorsOrigins = { "http://localhost:5002" },

                    AllowedScopes = { "openid", "profile", "api1" }
                }
            };
    }
}

ASP.NET API Service Startup.cs

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.IdentityModel.Tokens;
using Tps.ManagedClaimsWell.ApplicationServer.AppServInternals;
using Tps.ManagedClaimsWell.ApplicationServer.DataAccess;
using Tps.ManagedClaimsWell.ApplicationServer.Utility;

namespace ManagedClaimsWell.ApplicationServer.Core
{
    public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }

        public IConfiguration Configuration { get; }

        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            JwtSecurityTokenHandler.DefaultMapInboundClaims = false;

            services.AddControllers()
                .AddNewtonsoftJson();

            services.AddHttpClient();

            var appServSettings = new AppServSettings(Configuration);

            ClaimsWellCache.Inst.Load(ClaimsWellSchemaData.Load, IdentityData.UpdateNameLastAccessed);

            services.AddSingleton<IDiscoveryCache>(r =>
            {
                var factory = r.GetRequiredService<IHttpClientFactory>();
                return new DiscoveryCache(Constants.Authority, () => factory.CreateClient());
            });

            //services.AddAuthentication(options =>
            //{
            //    options.DefaultScheme = CookieAuthenticationDefaults.AuthenticationScheme;
            //    options.DefaultAuthenticateScheme = CookieAuthenticationDefaults.AuthenticationScheme;
            //    options.DefaultChallengeScheme = IISDefaults.AuthenticationScheme;
            //})
            //.AddCookie(CookieAuthenticationDefaults.AuthenticationScheme, options =>
            //{
            //    options.
            //    options.ExpireTimeSpan = TimeSpan.FromDays(1);
            //});

            services.AddAuthorization(options =>
            {
                options.AddPolicy("scope", policy =>
                {
                    policy.AddAuthenticationSchemes(CookieAuthenticationDefaults.AuthenticationScheme)
                        .RequireAuthenticatedUser()
                        .RequireClaim("scope", "api1");
                });
            });

            services.AddAuthentication(options =>
            {
                options.DefaultScheme = CookieAuthenticationDefaults.AuthenticationScheme;
                options.DefaultChallengeScheme = "oidc";
            })
            .AddCookie(options =>
            {
                options.Cookie.Name = "idsrv";
            })
            .AddOpenIdConnect("oidc", options =>
            {
                options.Authority = Constants.Authority;
                options.RequireHttpsMetadata = false;

                options.ClientId = "mcw.appserver";
                options.ClientSecret = "secret";

                // code flow + PKCE (PKCE is turned on by default)
                options.ResponseType = "code";
                options.UsePkce = true;

                options.Scope.Clear();
                options.Scope.Add("openid");
                options.Scope.Add("profile");
                options.Scope.Add("email");
                options.Scope.Add("api1");
                ////options.Scope.Add("transaction:123");
                ////options.Scope.Add("transaction");
                options.Scope.Add("offline_access");

                // not mapped by default
                options.ClaimActions.MapJsonKey(JwtClaimTypes.WebSite, "website");

                // keeps id_token smaller
                options.GetClaimsFromUserInfoEndpoint = true;
                options.SaveTokens = true;

                var handler = new JwtSecurityTokenHandler();
                handler.InboundClaimTypeMap.Clear();
                options.SecurityTokenValidator = handler;

                options.TokenValidationParameters = new TokenValidationParameters
                {
                    NameClaimType = JwtClaimTypes.Name,
                    RoleClaimType = JwtClaimTypes.Role,
                };
            });
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }

            app.UseHttpsRedirection();

            app.UseRouting();

            app.UseAuthentication();
            app.UseAuthorization();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers()
                    .RequireAuthorization();
            });
        }
    }
}

Answer 1

Don'tknow if its the issue but, one issue is that you have in IdentityServer

        app.UseIdentityServer();
        app.UseAuthorization();
        app.UseAuthentication();

See this article about how to configure the pipeline.

Especially, take notice about that it says:

UseIdentityServer includes a call to UseAuthentication, so it's not necessary to have both.

As I said in the comments, trying to send a request to /signin-oidc from postman will probably fail due to various built in features in how authentication works. One problem is that you don't have the correct state parameter that the OpenIdConnect handler expects. Its a random value that changes each time a user tries to authenticate.

Your "ASP.NET API Service Startup.cs", is a "client", not an API. What you have is meant for a end-user to login to. Usin postman here makes no sense. An API should probably use the UseJwtBearer handler instead and to that one you could send requests using PostMan and a valid access-token.