Question

Forms Notification Hub not working on SSL

Forms

Updated August 26, 2020

asked on September 15, 2018

Hi,

I am trying to get Forms Notification working on SSL. When I remove SSL settings from Forms, everything works fine, however, I am unable to get it working once adding SSL to the configuration.

I have followed the instructions listed here. All commands run successfully and config files are as they should be. I'm using a self generated certificate.

If I reset everything back to normal HTTP, everything works fine. I've also tried turning off firewalls, reboots and configuring on other Forms servers in case there's something up with this particular one, still the same end result.

If I look at the browser's console, I have this response on the GET request:

{"errors":[{"status":500,"code":"NotificationServerConnectionDown","message":"A connection to the notification server could not be made. Notifications and real-time updates have been disabled. [LFF8100-NotificationServerConnectionDown]"}]}

it also then says "Notifications have been disabled"

Forms Operation Event log reports the following:

A connection to the notification server could not be made. Notifications and real-time updates have been disabled. [LFF8100-NotificationServerConnectionDown]

Details:
URL: /Forms/webapi/v1/notification/GetPushNotificationToken?forceNew=false&skipErrMsg=true
Error: NotificationServerConnectionDown
Date: 9/15/2018 11:56:41 AM (South Africa Standard Time)
HTTP Status Code: 500
Business Process ID: 0
User: ADMIN
IP: 196.192.190.85
Browser: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36

Stack Trace:
Caught exception: Laserfiche.Forms.CommonUtils.Exceptions.LFFormsException
Message: A connection to the notification server could not be made. Notifications and real-time updates have been disabled. [LFF8100-NotificationServerConnectionDown]
   at E_Forms.WebApi.Version1.Controllers.NotificationController.GetPushNotificationTokenGeneral(PushServiceRegisterRequest registerReq, Boolean forceNew)
   at lambda_method(Closure , Object , Object[] )
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<GetExecutor>b__9(Object instance, Object[] methodParameters)
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<ExecuteActionFilterAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<ExecuteActionFilterAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ExceptionFilterResult.<ExecuteAsync>d__0.MoveNext()

I've run out of ideas on what to look at next, hoping you guys can assist me.

Thanks

Sheldon

5 0

Answer

SELECTED ANSWER

replied on December 14, 2018

We just ran into the same issue, and the solution was indeed the steps listed at:

https://www.laserfiche.com/support/webhelp/Laserfiche/10/en-US/administration/#../Subsystems/Forms/Content/Notification-Service.htm

Specifically, running these commands:

netsh http add sslcert ipport=0.0.0.0:8181 certhash=#certhash appid={#appid}

netsh http add urlacl url=https://*:8181/ user="LOCAL SERVICE" listen=yes

Some additional helpful detail:

BEFORE running the first command, stop the "Laserfiche Notification Hub Service". This is because when the service starts, it binds to TCP 8181 for http traffic. The first command sets a binding/listener for TCP 8181 for httpS traffic... it fails to do this if the Hub service is running, because 8181 is already in use. You have to stop the service so that listener drops.
BEFORE running the first command, run "netsh http show iplisten", you'll see all http listeners. You shouldn't see any for port 8181.
RUN the first command...
- When running the first command, your certificate must be installed on the system first, and pull the cert hash from Certificates MMC or the IIS MMC > Server Certificates applet....
- For #appid, generate a random guid, using a tool like: https://www.guidgenerator.com/
AFTER running the first command, rerun "netsh http show iplisten", you should see your listener on 8181 now.
The second command then registers the URL for the 8181 traffic with the http service... basically saying, "ok, any http(s) request that comes in bound for a url of https://*:8181 needs to be routed to the Hub service, not to the Web Service (or any other application leveraging the https service/api)".
BEFORE running the second command, run "netsh http show urlacl | findstr :8181", it should return nothing (assuming the URL wasn't already registered.
RUN the second command, replacing LOCAL SERVICE with whatever identity your "Laserfiche Notification Hub Service" is running under... by default, it's the LOCAL SYSTEM account, which you actually have to specify in a different syntax than just "LOCAL SYSTEM"... the command would actually be:
- netsh http add urlacl url=https://*:8181/ user="NT AUTHORITY\SYSTEM" listen=yes
AFTER running that command, rerun "netsh http show urlacl | findstr :8181", and it should now return:
- Reserved URL : http://+:8181/
Go into your Forms Config, the Notification Service and ensure:
1. the Use SSL option is enabled
2. You're using a FQDN in the URL and it matches the name in the certificate you used in the first command. I.e. URL is "myserver.mydomain.com:8181" and certificate is "myserver.mydomain.com" or "*.mydomain.com"
Start the Laserfiche Notification Hub Service (or restart if you'd already had it running)
Restart the Laserfiche Notification Master Service

And for us, we were rolling.

3 0

replied on February 27, 2019

Hi Damon,

I'm trying this configuration again on another server and following your instructions listed here. When I reach step 4, the IP listen list is empty. Do you know perhaps what I can do to rectify this? The command says it added successfully and the service was stopped at the time.

0 0

replied on August 26, 2020

Apparently, if all bindings (whether specified through IIS Manager on your website bindings or via the "netsh http add") for the http service are configured to listen on all IP addresses (i.e. 0.0.0.0 or displays as * in IIS Mgr) then they don't show up in the "netsh http show iplisten" output, because it considers that to be it's default approach.

They'll only show up if there is a non-default iplistener implemented, i.e., a listener on a specific IP address:

netsh http add sslcert ipport=10.1.1.26:8181 certhash=#certhash appid={#appid}

but that's okay, there's a couple other ways to verify:

if you run the command a second time, it will tell you the listener already exists (the message is something like "file exists").

Or, you can use netstat to look at the listeners on PID 4, which is always the SYSTEM process, and includes httpsys as it's a kernel mode driver.

netstat -ano | findstr /I listen

My screen capture here is NOT from a Forms server, so I don't have the :8181 listener... it's just meant to illustrate identifying the http listeners that are actively running under the system process.

0 0

Replies

replied on September 17, 2018

I think the instructions on the laserfiche site forget a step. I think you also have to use the netsh commands under step 2 to bind the certificate to the Master service on port 8268 (just use the same commands, but replace references to 8181 with 8268). What is happening is that when a client connects to their Inbox, a request is made to the forms server to obtain a token from the push notification service. With SSL enabled, this causes the Forms IIS application to make a SSL encoded request to the Master Notification Service for a token. This is the step that is failing for you, because the service on 8268 is not bound to a certificate and cannot establish https connections.

0 0

View 3 previous replies

replied on September 17, 2018

Hi Scott,

Thank you for the feedback. Your explanation makes sense and I ran the two netsh commands again for port 8268 and it ran successfully. However, after restarting all services, the same issue still remains.

It's odd that they forgot that step in the help file, yet FormsConfig doesn't offer SSL settings for this particular setting so perhaps it's not necessary at all?

0 0

replied on September 17, 2018

OK, you're right. I looked back at my notes and it was indeed only the 8181 that needed the binding. Is your forms server behind any complex network setup such as a reverse-proxy configuration? Do you have an external/internal forms setup with the notification services in a different location than your forms server?

The error you posted is strongly suggests a problem with the connection between the Forms IIS application and the master notification service endpoint.

0 0

replied on September 17, 2018

I believe that network and software setup is acceptable since the simple removal of SSL allows notification services to work as expected.

Forms and Notification services are installed on the same machine and it's the only one configured. Even localhost connection disables the notification services if that helps paint the picture. No proxies configured at all.

If the issue is Forms Application through IIS not able to connect to Master Notification Service, I've also eliminated local Firewall from being the issue by disabling it during testing. Once again, without SSL it's fine, with SSL it won't connect. I'm sure your previous statement about binding might be part of the issue.

0 0

replied on September 17, 2018

What do you have for the setting "Laserfiche Notification Service Server" on your FormsConfig page under the Laserfiche tab? Also, I don't think it's a binding issue because according to the config files, the master service on 8268 is just using a NET.TCP binding, not HTTP protocol.

0 0

replied on September 17, 2018

Here's the current settings. "Rogue" is the Forms server name.

0 0

replied on September 17, 2018

Hmm, maybe try using the FQDN there, and also make sure you have the FQDN (matching the domain from your SSL Cert) for the Primary Forms Server URL also.

1 0

replied on December 12, 2018

Hi everyone.

Did you manage to get a solution for this problem by any chance?

I am receiving the exact same problem. I have noticed the issue on a bit more of a severe note however. My Forms installation completely stopped working this morning and I am presuming it has something to do with this error:

When connecting to the localhost/Forms it spat out a 404 error.

After further investigation, I have noticed that every time an end user connects on their machines to submit a Form, this error prompts:

A connection to the notification server could not be made. Notifications and real-time updates have been disabled. [LFF8100-NotificationServerConnectionDown]

Details:
URL: /Forms/webapi/v1/notification/GetPushNotificationToken?forceNew=false&skipErrMsg=true
Error: NotificationServerConnectionDown
Date: 12/13/2018 8:58:58 AM (South Africa Standard Time)
HTTP Status Code: 500
Business Process ID: 0
User: Cheryl Hoffmann
IP: 192.168.55.71
Browser: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36

Stack Trace:
Caught exception: Laserfiche.Forms.CommonUtils.Exceptions.LFFormsException
Message: A connection to the notification server could not be made. Notifications and real-time updates have been disabled. [LFF8100-NotificationServerConnectionDown]
   at E_Forms.WebApi.Version1.Controllers.NotificationController.GetPushNotificationTokenGeneral(PushServiceRegisterRequest registerReq, Boolean forceNew)
   at lambda_method(Closure , Object , Object[] )
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<GetExecutor>b__9(Object instance, Object[] methodParameters)
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<ExecuteActionFilterAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<ExecuteActionFilterAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ExceptionFilterResult.<ExecuteAsync>d__0.MoveNext()

I have looked at the thread you linked when posting your question @████████https://www.laserfiche.com/support/webhelp/Laserfiche/10/en-US/administration/#../Subsystems/Forms/Content/Notification-Service.htm and am curious what your config is in the Laserfiche Notification Hub Service and Master Service?

Question

Question

Forms Notification Hub not working on SSL

Answer

Replies

Sign in to reply to this post.