I gave it a shot and I still get the popup. Whats really weird is that if I close the popup and get this screen,

the Windows Authentication button doesn't do anything. I have changed almost all the application pool identity to my server account I use for everything LF. 

I keep getting this in the event viewer for WebSTS >Operations.

ID4243: Could not create a SecurityToken. A token was not found in the token cache and no cookie was found in the context.

System.IdentityModel.Tokens.SecurityTokenException: ID4243: Could not create a SecurityToken. A token was not found in the token cache and no cookie was found in the context.
   at System.IdentityModel.Tokens.SessionSecurityTokenHandler.ReadToken(XmlReader reader, SecurityTokenResolver tokenResolver)
   at System.IdentityModel.Tokens.SessionSecurityTokenHandler.ReadToken(Byte[] token, SecurityTokenResolver tokenResolver)
   at System.IdentityModel.Services.SessionAuthenticationModule.ReadSessionTokenFromCookie(Byte[] sessionCookie)
   at System.IdentityModel.Services.SessionAuthenticationModule.TryReadSessionTokenFromCookie(SessionSecurityToken& sessionToken)
   at System.IdentityModel.Services.SessionAuthenticationModule.OnAuthenticateRequest(Object sender, EventArgs eventArgs)
   at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

If you can, I'd try uninstalling and reinstalling LFDS/STS under the default Network Service identity. This won't affect the LFDS database and it should automatically re-connect because uninstalling doesn't remove the LFDS SQL connection profile which resides in a config file in \ProgramData\Laserfiche\*.

Hi Samuel. I tried this already. Did that this weekend when no one was on the system. Still have the issue...

Hi Lucas, it looks like we need to do some deeper investigation so I recommend having your SP open a support ticket with our Support team for this issue.

Thanks Chase, I did open a case. 

My network admin ended up finding the fix. It was in the provider settings for Windows Authentication in IIS. As soon as we switched it to Negotiate first, then NTLM, it started working. The install for LF 11 must have switched it to defaults. All the documentation that LF provides shows NTLM first and Negotiate second so I never thought to change it myself. 

Hi Lucas, Laserfiche actually doesn't control the priority of enabled Windows Authentication providers, as that is a system setting. I am not sure how automatic Windows Authentication was working before with your providers out of order but I'm glad it's working now with the correct settings as you show in your screenshot!