Thanks Samuel - that's the nudge I needed in the right direction. I kept wanting to have a 2nd instance on the same server.

Follow-up question.  My original WebSTS is located on the Directory Server and is working appropriately (allowing both LF & Windows logins).  The new WebSTS is located on a different web server from the Directory Server and is configured to force Windows Auth.  However I'm getting the following error message:
"Windows Authentication is not available. Please type your Windows credentials below or try another login method."  Do I need to configure delegation for this to work or is this telling me something else?

Hey John, glad you found the links helpful.

I'm guessing you have your "WebLink STS" configured using the Alternate Service. The Alternate Service uses certificate authentication and does not support Windows Authentication. As long as the "WebLink STS" instance is on a domain-joined server, you can use the normal method.

If any of that is confusing, post a screenshot of your LFDS XML and STS Endpoint Configs and I can give more specific direction.

Not using the alternate service on either instance.

I'm getting the following in the Event Viewer for WebSTS/Operations:

Log Name:      Laserfiche-Directory Service-WebSTS/Operational
Source:        Laserfiche-Directory Service-WebSTS
Date:          3/31/2020 2:29:48 PM
Event ID:      0
Task Category: None
Level:         Information
Keywords:      
User:          NETWORK SERVICE
Computer:      ******************************
Description:

Laserfiche.IdentityModel.UnknownIdentityProviderErrorException ---> Laserfiche.LicenseManager.LMO.LMOException: Exception of type 'Laserfiche.LicenseManager.InvalidWindowsAuthenticationException' was thrown.
   at Laserfiche.LicenseManager.LMO.SessionTokenFactory.Login2(Database database, String xmlRST, Dictionary`2 paramBag)
   at Laserfiche.IdentityModel.LFDSIdentityService.GetBearerToken2(Dictionary`2 loginParameters)
   --- End of inner exception stack trace ---
   at Laserfiche.IdentityModel.LFDSIdentityService.ConvertLmoException(LMOException ex)
   at Laserfiche.IdentityModel.LFDSIdentityService.GetBearerToken2(Dictionary`2 loginParameters)
   at WebSTS.LFDS.Services.Login.LFDSLoginManager.ProcessLogin(LoginData loginData)
   at WebSTS.LFDS.Controllers.LoginController.Login(LoginData data)
   at lambda_method(Closure , Object , Object[] )
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<GetExecutor>b__9(Object instance, Object[] methodParameters)
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ExceptionFilterResult.<ExecuteAsync>d__0.MoveNext()

I've attached screenshots of the XML & STS Endpoint utilities.  The STS Endpoints look exactly the same on both the LFDS Server (this one works), and the secondary STS server (this one doesn't work).

Thanks John. Spoke with some folks internally. As you're likely aware, LFDS/STS 10.4.3 brought an update to how those services communicate. One consequence is that separate STS instances on the same domain now require configuring Kerberos delegation to the LFDS instance in order to support Windows Authentication. We unfortunately do not have step-by-step documentation at this time.

Are you familiar with Kerberos delegation at all?

Edit: Towards the top of the LFDS 10.4.3 Known Issues list is:

The Windows Authentication button on the Directory Server sign-in page may be hidden by default when Directory Server and the STS are installed on separate computers. An administrator can show the Windows Authentication button by navigating to the STS configuration page and clearing the Hide Windows Authentication check box. Please note that when Directory Server and the STS are on separate computers, proper Kerberos configuration will be required for the Windows Authentication button to work properly. (208130)

Thanks Samuel - that's what I suspected.

I am a little familiar with delegation - do you have any guidelines specific to configuring the "remote" WebSTS?  I've been playing with configuring delegation on the WebSTS server (front-end delegation), but I'm not sure which Service Type to allow going to the LFDS server.

If I recall correctly the Service Type is "http".

I had no luck with configuring it via the A.D. UI for the front-end server (even allowing all services), so I tried to do it for "resource based constrained delegation" using the following from the "Configuring Kerberos for Laserfiche 10 Web Products in a Windows Server 2016 and IIS 10 Environment" white-paper:

...

If the front-end service uses a computer account and the back-end service uses a domain user account:

$frontendidentity = Get-ADComputer –Identity WebServer
$backendidentity = Get-ADUser –Identity LaserficheServerDomainAccount
Set-ADUser $backendidentity –PrincipalsAllowedToDelegateToAccount
$frontendidentity

Still no luck....

Hi John,

I got Windows Auth working through a separate WebSTS instance in my test environment today. I used the default machine identities for both LFDS and STS, so it's possible you'll need extra steps for setting SPNs for the Kerberos delegation config to work.

Turns out there's one more quirk of Kerberos delegation I didn't know about. It seems to only work when the site you're accessing is in the "Local Intranet" security zone, configured under the client machine's Internet Properties > Security settings.

When you click the blue Windows Authentication button on the STS login page, do you get a popup asking for your credentials like so?

That indicates that the site is not in a zone configured for automatic pass-through authentication. After entering my credentials, I got the same error you did.

Once I got my STS site mapped to the Local Intranet zone, clicking the Windows Auth button automatically passed my credentials and successfully logged me into Laserfiche.

The security zone settings must be set for all client machines accessing the STS instance. It appears simple to push them out via GPO to end users if necessary: Securing zone levels in Internet Explorer (Chrome/Edge read the same settings).

I've gotten it working with the delegation configuration below, however this isn't exactly ideal...

The LFDS service on the Directory Server is running under a domain account so I created SPNs for that Service Login for HTTP (and LaserficheServer as a long shot per the documentation), however that's not working (below)

I'm not sure where to go from here...