We're testing the self-hosted LF API Server, and it works fine when the connection to the Repository Server does NOT use SSL. However when SSL is enabled for the connection (from API to LFS), we receive the following response:
{ "type": "accessDenied", "title": "Error: No Laserfiche Repository connection could be made.", "status": 401, "instance": "/v1/Repositories/MCCi-Test/Token", "operationId": "95a8eb5e57194b8b9babcaedcf2c01a8", "errorSource": "Api Server", "errorCode": 201, "traceId": "00-75f5b72d1734abb38240b1ea6f1f6a96-b05954a36463887e-00" }
Digging into the LFRepositoryAPI operation log, we see it fails due to certificate validation:
[95a8eb5e57194b8b9babcaedcf2c01a8]:LFRepositoryAPI started at 2023-01-06T15:40:46.3047858Z.
[INFO+0ms(3)]Correlated operation: 80000058-0001-f800-b63f-84710c7967bb
[INFO+0ms(3)][POST] https://mccitestpro.mccicloud.io/lfrepositoryapi/v1/Repositories/MCCi-Test/Token
[INFO+0ms(3)]Fail to get Authorization header from context
[INFO+0ms(3)]Fail to get Authorization header from context
[INFO+152ms(3)]Failed to log in with the provided credentials.
[ERR+175ms(3)]One or more errors occurred. (One or more errors occurred. (The SSL connection could not be established, see inner exception.))
THE SSL CONNECTION COULD NOT BE ESTABLISHED, SEE INNER EXCEPTION.
THE REMOTE CERTIFICATE WAS REJECTED BY THE PROVIDED REMOTECERTIFICATEVALIDATIONCALLBACK.
-----TRACES----
System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: The remote certificate was rejected by the provided RemoteCertificateValidationCallback.
at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
at System.Net.Security.SslStream.CompleteHandshake(SslAuthenticationOptions sslAuthenticationOptions)
at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at Laserfiche.HttpClient.TimeoutHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
------
[INFO+178ms(3)]TokenController GetAccessToken failed for MCCi-Test
[ERR+195ms(3)]Error: No Laserfiche Repository connection could be made.
ERROR: NO LASERFICHE REPOSITORY CONNECTION COULD BE MADE.
-----TRACES----
Laserfiche.Repository.LFSConnector.Exceptions.AccessDeniedException: Error: No Laserfiche Repository connection could be made.
at Laserfiche.Repository.LFSConnector.Connectors.RepositoryConnector.CreateConnection_V1(String repositoryId, CreateConnectionRequest request, Boolean createCookie)
at Laserfiche.Sites.SiteApiRepository.V1.Controllers.TokenController.<>c__DisplayClass3_0.<<GetAccessToken>b__0>d.MoveNext() in E:\A\1261\s\src\V1\Controllers\TokenControllerV1.cs:line 85
--- End of stack trace from previous location ---
at Laserfiche.Sites.SiteApiRepository.Polly.PollyExecutor.<>c__DisplayClass2_0`1.<<ExecuteAsync>b__0>d.MoveNext() in E:\A\1261\s\src\Polly\PollyExecutor.cs:line 31
--- End of stack trace from previous location ---
at Polly.CircuitBreaker.AsyncCircuitBreakerPolicy.<>c__DisplayClass8_0`1.<<ImplementationAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Polly.CircuitBreaker.AsyncCircuitBreakerEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext, ExceptionPredicates shouldHandleExceptionPredicates, ResultPredicates`1 shouldHandleResultPredicates, ICircuitController`1 breakerController)
at Polly.CircuitBreaker.AsyncCircuitBreakerPolicy.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
at Polly.AsyncPolicy.ExecuteAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
at Laserfiche.Sites.SiteApiRepository.Polly.PollyExecutor.ExecuteAsync[T](Func`1 action, ISparkContext sparkContext) in E:\A\1261\s\src\Polly\PollyExecutor.cs:line 28
at Laserfiche.Sites.SiteApiRepository.Controllers.ODataBaseController.<>c__DisplayClass5_0`1.<<TryCatchAsync>b__0>d.MoveNext() in E:\A\1261\s\src\Controllers\ODataBaseController.cs:line 51
--- End of stack trace from previous location ---
at Laserfiche.Sites.SiteApiRepository.Util.ControllerUtil.TryCatchAsync[T](ControllerBase controller, ISparkContext sparkContext, Func`1 p, Action`1 resultLogAction, Func`2 customResponseHandling) in E:\A\1261\s\src\Util\ControllerUtil.cs:line 61
------
-----DATA-----
OperationId = 95a8eb5e57194b8b9babcaedcf2c01a8
[INFO+213ms(3)]HTTP Reponse: 401.
[INFO+213ms(3)]Completed at 2023-01-06T15:40:46.5182802Z MilliSpeedOp.
[LFRepositoryAPI]Error: No Laserfiche Repository connection could be made.[95a8eb5e57194b8b9babcaedcf2c01a8][2!0](213ms)>[Warning][-1][80000058-0001-f800-b63f-84710c7967bb][system]
We're enabling/disabling the use of SSL/TLS to the repo server by adjusting the following parameter in the appsettings.json:
- "EnableLaserficheServerSSL": true,
The environment:
- WebServer 10.20.11.4 (mcmccitestLF01.mccicloud.io): Web Client, Weblink, LF API Server
- AppServer 10.20.11.36 (mcmccitestLF02.mccicloud.io): LF Server (with repository “mcci-test”)
For this connection, the SSL client would be 10.20.11.4 and the SSL server would be: 10.20.11.36. The Certificate Validation would be performed by the LFAPI on 10.20.11.4.
Interestingly enough, we also have Web Client and Weblink on the WebServer, both of those configured to connect using SSL/TLS to the same repository on the AppServer. Both connect successfully without any certificate validation errors. Additionally, connecting from browser on 10.20.11.4 to 10.20.11.36 using HTTPS, the browser indicates the same certificate is passing validation.
We've also manually validated the certificate via the standard validation criteria:
- hostname used for connection (mcmccitestLF02.mccicloud.io) matches the common name on the certificate (*.mccicloud.io).
- current date is within the certificates valid from and valid to dates
- sha2 signed
- the SSL client (10.20.11.4) can reach the CRL list site and successfully retrieve the CRL.
- the certificate is not on the CRL list.
- the certificate chain of trust is intact and the root certificate is in the Trusted Root Certificates cert store on the the SSL client system (10.20.11.4) (this is a godaddy issued certificate). intermediate certificate is available on the SSL server system.
Monitoring the communication with wireshark provides no additional insight... the SSL handshake proceeds normally, and then upon completion the SSL Client (10.20.11.4) sends a communication to terminate the session. This is seen as the Encrypted Alert packet in the capture. Presumably, it does this because at that stage, while the SSL session has been negotiated, this is where the SSL client (LFAPI) has performed the cert validation, and has deemed it "not valid", so the session is terminated. Below is that capture of the LFAPI connecting to the LFS Repo.
And the following is the Web Client on the same WebServer connecting to the same LFS Repo. Notice, it completes exactly the same as above, except there's no "encrypted alert" and the conversion progresses successfully beyond the end of the handshake.
The only thing we found via wireshark is when the LFAPI does the CLIENT HELLO portion of the SSL handshake, it erroneously puts the REPO name in the SSI value.
Here is the CLIENT HELLO (as captured by wireshark) with a correct value when the connection is initiated by the Web Client:
And here is the CLIENT HELLO (as captured by wireshark) with the incorrect value when the connection is initiated by the LFAPI:
We're sending this in to Support today as a bug.
Based on our particular bindings, this doesn't create a problem with making the connection, IIS serving the correct site and certificate (as their is only 1 site, and 1 https binding and it has no SNI enabled). However, it may still point to the underlying issue... Either:
a) the LF API is using a different certificate validation process that's finding some validation problem that Web Client and Weblink's connections do not find.
OR
b) this incorrect hostname value showing in the SNI parameter is indicative of the wrong hostname ALSO being used in the name match evaluation of the certificate validation process.
I'd be surprised no one else ran into this, unless everyone isn't using SSL on the connection from the API to LFS. Has anyone successfully exercised the self-hosted API with the "EnableLaserficheServerSSL" setting to true?
Based on the swagger page, we're Build# : 1.0.0.41.