Forms authentication machineKey "unable to validate data" error in SSRS

Question

I have followed the guides here to setup a custom security extension in SSRS. So far the login and ReportServer are working however the Report portal https://localhost/reports has a 500 error. From the logs:

2022-11-10 20:45:02.8111|INFO|1|File Logger created: C:\Program Files\Microsoft SQL Server Reporting Services\SSRS\LogFiles\RSPortal_2022_11_10_20_45_02.log - level Info, will roll at 32 Mb, process id 32592
2022-11-10 20:45:02.8391|INFO|1|Provider name 
2022-11-10 20:45:02.8391|INFO|1|Container name Microsoft SQL Server Reporting Services Key Container 2010
2022-11-10 20:45:02.8391|INFO|1|Setting Symmetric Key
2022-11-10 20:45:02.8548|INFO|1|Setting up Hosted Process State
2022-11-10 20:45:03.1990|INFO|1|Starting ReportServerWebApp
2022-11-10 20:45:03.3871|INFO|1|Working directory : C:\Program Files\Microsoft SQL Server Reporting Services\SSRS\Portal
2022-11-10 20:45:03.3871|INFO|1|Report Server url: http://performa-pc/ReportServer
2022-11-10 20:45:03.3871|INFO|1|Report Server Web App virtual root: /Reports
2022-11-10 20:45:03.3871|INFO|1|Web app url: http://+:80/Reports/
2022-11-10 20:45:03.3871|INFO|1|Web app url: https://Performa-PC:443/Reports/
2022-11-10 20:45:03.3871|INFO|1|Authentication scheme(s): Anonymous
2022-11-10 20:45:13.5444|INFO|7|Received request GET | RequestID = s_d51a836d-7628-4ab2-b43a-cf3bcab5af56 
2022-11-10 20:45:13.5921|ERROR|7| 192.168.59.181: GET  - 0:00:00.0518145
Exception: System.Web.HttpException (0x80004005): Unable to validate data.
   at System.Web.Configuration.MachineKeySection.EncryptOrDecryptData(Boolean fEncrypt, Byte[] buf, Byte[] modifier, Int32 start, Int32 length, Boolean useValidationSymAlgo, Boolean useLegacyMode, IVType ivType, Boolean signData)
   at System.Web.Security.FormsAuthentication.Decrypt(String encryptedTicket)
   at Microsoft.BIServer.Owin.Common.Middleware.CustomAuthenticationMiddleware.CreateRequestContextFromCookie(IOwinContext context)
   at Microsoft.BIServer.Owin.Common.Middleware.CustomAuthenticationMiddleware.CreatePortalIdentity(IOwinContext context)
   at Microsoft.BIServer.Owin.Common.Middleware.CustomAuthenticationMiddleware.Invoke(IOwinContext context)
   at Microsoft.Owin.Security.Infrastructure.AuthenticationMiddleware`1.<Invoke>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.BIServer.Owin.Common.Middleware.RequestLoggingMiddleWare.<Invoke>d__2.MoveNext()| RequestID = s_d51a836d-7628-4ab2-b43a-cf3bcab5af56 

Apparently it's having trouble with the machineKey used for encryption.

I have included the machineKey in:

• web.config
• rsReportServer.config
• rsPortal.exe.config

<machineKey validationKey="33A11FDECC5CB917123E44C5BDAF1859942D5FD0D80E1CA3FF2F32576F391556" decryptionKey="1581FF2A206D1A3C283442C84EF2EBD333AE0B27BC85C502E1A771058539C4B1" validation="AES" decryption="AES" />

Any suggestions as to what I could be missing?

UPDATE:

SQL Management Studio Logs:

11/10/2022 18:16:45,.NET Runtime,Warning,Category: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager<nl/>EventId: 35<nl/><nl/>No XML encryptor configured. Key {2303a688-2964-441a-bc7a-fd28a9da1f19} may be persisted to storage in unencrypted form.,(0),1000,,Test-PC
11/10/2022 18:16:45,.NET Runtime,Warning,Category: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager<nl/>EventId: 59<nl/><nl/>Neither user profile nor HKLM registry available. Using an ephemeral key repository. Protected data will be unavailable when application exits.,(0),1000,,Test-PC
11/10/2022 18:16:45,.NET Runtime,Warning,Category: Microsoft.AspNetCore.DataProtection.Repositories.EphemeralXmlRepository<nl/>EventId: 50<nl/><nl/>Using an in-memory repository. Keys will not be persisted to storage.,(0),1000,,Test-PC
11/10/2022 18:16:20,Microsoft-Windows-HttpService,Information,Attempted to add URL (http://+:80/Reports/) to URL group (0xFE00000420000002). Status: 0x0. Process Id 0x6978 Executable path \Device\HarddiskVolume3\Program Files\Microsoft SQL Server Reporting Services\SSRS\Portal\RSPortal.exe<c/> User S-1-5-80-4050220999-2730734961-1537482082-519850261-379003301,(5),113,NT SERVICE\SQLServerReportingServices,Test-PC
11/10/2022 18:16:20,Microsoft-Windows-HttpService,Information,Attempted to add URL (https://Test-PC:443/Reports/) to URL group (0xFE00000420000002). Status: 0x0. Process Id 0x6978 Executable path \Device\HarddiskVolume3\Program Files\Microsoft SQL Server Reporting Services\SSRS\Portal\RSPortal.exe<c/> User S-1-5-80-4050220999-2730734961-1537482082-519850261-379003301,(5),113,NT SERVICE\SQLServerReportingServices,Test-PC
11/10/2022 18:16:20,Microsoft-Windows-HttpService,Information,Create URL group 0xFE00000420000002. Status 0x0. Process Id 0x6978 Executable path \Device\HarddiskVolume3\Program Files\Microsoft SQL Server Reporting Services\SSRS\Portal\RSPortal.exe<c/> User S-1-5-80-4050220999-2730734961-1537482082-519850261-379003301,(5),111,NT SERVICE\SQLServerReportingServices,Test-PC
11/10/2022 18:16:19,Microsoft-Windows-HttpService,Information,Attempted to add URL (https://Test-PC:443/ReportServer/) to URL group (0xFC00000620000002). Status: 0x0. Process Id 0x10C4 Executable path \Device\HarddiskVolume3\Program Files\Microsoft SQL Server Reporting Services\SSRS\ReportServer\bin\ReportingServicesService.exe<c/> User S-1-5-80-4050220999-2730734961-1537482082-519850261-379003301,(5),113,NT SERVICE\SQLServerReportingServices,Test-PC
11/10/2022 18:16:19,Microsoft-Windows-HttpService,Information,Attempted to add URL (http://+:80/ReportServer/) to URL group (0xFC00000620000002). Status: 0x0. Process Id 0x10C4 Executable path \Device\HarddiskVolume3\Program Files\Microsoft SQL Server Reporting Services\SSRS\ReportServer\bin\ReportingServicesService.exe<c/> User S-1-5-80-4050220999-2730734961-1537482082-519850261-379003301,(5),113,NT SERVICE\SQLServerReportingServices,Test-PC
11/10/2022 18:16:16,Microsoft-Windows-HttpService,Information,Attempted to add URL (http://+:8082/) to URL group (0xFE0000002000001D). Status: 0x0. Process Id 0x2DFC Executable path \Device\HarddiskVolume3\Program Files\Microsoft SQL Server Reporting Services\SSRS\Management\RSManagement.exe<c/> User S-1-5-80-4050220999-2730734961-1537482082-519850261-379003301,(5),113,NT SERVICE\SQLServerReportingServices,Test-PC
11/10/2022 18:16:16,Microsoft-Windows-HttpService,Information,Create URL group 0xFE0000002000001D. Status 0x0. Process Id 0x2DFC Executable path \Device\HarddiskVolume3\Program Files\Microsoft SQL Server Reporting Services\SSRS\Management\RSManagement.exe<c/> User S-1-5-80-4050220999-2730734961-1537482082-519850261-379003301,(5),111,NT SERVICE\SQLServerReportingServices,Test-PC
11/10/2022 18:13:58,Microsoft-Windows-WAS,Information,A worker process with process id of '12564' serving application pool 'Saturn-AuthServiceAppPool' was shutdown due to inactivity.  Application Pool timeout configuration was set to 20 minutes.  A new worker process will be started when needed.,(0),5186,,Test-PC

I've tried many solutions aimed at addressing the machineKey error. Now I found the code for MachineKeySection.cs from Microsoft and it seems the "Unable_To_Validate_Data" error could be masking something else gone wrong....

// It's important that we don't propagate the original exception here as we don't want a production
// server which has unintentionally left YSODs enabled to leak cryptographic information.
            throw new HttpException(SR.GetString(SR.Unable_to_validate_data));

UPDATE 2:
I started debugging System.Web.dll source code. It looks like it is indeed a problem with the MachineKey. Will continue debugging tomorrow.

Answer 1

I found the issue. The Forms Authentication ticket had never been encrypted when it was created on the ReportServer app and so failed while trying to decrypt it on the ReportPortal app.

To fix this I enabled <forms... protection="All"> on the ReportServer web.config file so it encrypts the ticket.