You are viewing limited content. For full access, please sign in.

Question

Question

LF Cloud remote agent IP address changed

Laserfiche Cloud Rules Integrations Connector
Updated November 1, 2023
asked on October 26, 2023

We are using Laserfiche remote agent in  a rule from a business process to return data from an azure sql server on a VM.  Azure includes a security feature, there is an allowlist to name the IP address or range where the network traffic can originate from.  We have had the function operating for a year but this week it failed.

It seems the Laserfiche IP address changed from 179.61.228.117 to 179.61.228.141. Is this expected, does LF change the IP from time to time?  Is there a way to have a static ip address or is there a range where we can configure the firewall correctly? If LF address changes, we need to reconfigure the firewall and this is going to make the process inherently unreliable.  What is best practise for the set up of a firewall with remote agent requests?

0 0

Answer

APPROVED ANSWER
replied on October 27, 2023 • Show version history

Hi Sean,

The Laserfiche Cloud inbound endpoints are documented here:

https://doc.laserfiche.com/laserfiche.documentation/en-us/Default.htm#Software_Requirements.htm

The Remote Agent endpoints for each Laserfiche Cloud region are:

  • USA: bpm1.laserfiche.com
  • Canada: bpm1.laserfiche.ca
  • EU: bpm1.eu.laserfiche.com

We aim to have the Software Requirements documentation page updated with those soon.

The Laserfiche Cloud Server Endpoints section specifies the URLs for traffic initiated inbound to Laserfiche Cloud from clients. Remote Agents initiate an HTTPS connection to Laserfiche Cloud and then keep the connection open.

  • The lf-cloud-production-*-webaccess-datastore.s3.$region.amazonaws.com URLs are for Amazon's S3 object storage service. S3 is a shared service made up of thousands of underlying nodes. Laserfiche has no control over the S3 IP addresses Amazon uses. As these S3 endpoints are used with the Laserfiche Web Client (and thus interactive end users), they are not relevant to Remote Agents.
  • The *.lfxstatic.com entry covers CDN endpoints that serve various static assets, like web application HTML/CSS/image files. They are not relevant to Remote Agents.
  • The public-laserfichelocalhost-certificate.s3-us-west-1.amazonaws.com/lflocalhost.pfx entry covers a TLS certificate we provide that's required for the end user Laserfiche WebTools Agent / Scanning / Microsoft Office Integration utilities. These utilities fetch the latest available certificate file (1-year validity period) from that URI. They are not relevant to Remote Agents.

The Laserfiche Cloud IP Addresses section defines the IPs that Laserfiche Cloud uses for outbound connections through the Web Request Rule and Application Connections features. These are not applicable to Remote Agent communications because Laserfiche Cloud does not initiate outbound calls to Remote Agents.

These two categories are completely different and unrelated from a networking perspective. 

Currently, IP addresses for inbound endpoints, including those Remote Agents communicate with, are subject to change. Inbound connectivity requirements are expressed in terms of hosts/domains that must be allowed, such as "*.laserfiche.com". Restricting your outbound network traffic based on destination hostname generally requires either:

  • A more advanced firewall with FQDN filtering capabilities, such as Azure Firewall. 
  • A forward (outbound) HTTP proxy like Squid Proxy (a popular option) or this new Azure Firewall Explicit Proxy feature that you configure allowed hosts/domains for and route your outbound traffic through.

We understanding that not having static IPs (or at least a consistent IP range) for Laserfiche Cloud's inbound Remote Agent endpoint makes network security configuration more complicated for our customers. We're discussing internally and will at minimum strive to make our documentation more clear on recommended practices and requirements.

As an alternative option, you could write a PowerShell script that checks the public DNS entry for the Remote Agent endpoint (bpm.laserfiche.com? - not 100% sure offhand) and using the Azure PowerShell modules programmatically updates the relevant Azure Network Security Group firewall rule if/when the IP address changes. Have that run every five minute schedule somewhere, like via Windows Task Scheduler on the Remote Agent VM or through Azure Automation.

2 0

Replies

replied on October 30, 2023

Thanks for the response, Simon, i will seek some assistance to see if a dynamic allowlist assignment is possible but would note the IP address range that is originating from Laserfiche is outside the range that is published on the link.  We see 179.61.228.141 and the reference article lists

52.60.44.193, 3.97.48.213, 3.96.88.70 for the Canadian data center

 

0 0
replied on October 30, 2023

Further, do you know what is the FQDN where remote agent requests originate from the Canadian data centre? If I read the documentation here, I see lf-cloud-production-ca-central-1-webaccess-datastore-4053.s3.ca-central-1.amazonaws.com (for canada) resolves to 52.95.190.54 and that is outside the range of IP's in the online documentation and it's also very different to what we're seeing in production 179.61.228.141. I like the powershell approach suggested but would need to know how to obtain the current IP address to make the allowlist dynamic?

 

0 0
replied on October 31, 2023 • Show version history

I've updated my original reply with the Remote Agent endpoints for each region along with some further clarifications.

------

As an aside, it's helpful if you Reply to the comments (even your own) so they properly thread, rather than Replying to the post. Makes conversations much easier to follow.

0 0
replied on October 30, 2023

More| Ive just retested again and its broken again because the IP address is something else entirely.  Client with IP address '182.18.213.183' is not allowed to access the server. To enable access, use the Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range.

 

how often it this address expected to change? We've had this is production for months, why has it just started to change IP's now?

 

0 0
replied on October 31, 2023 • Show version history

Sean,

Remote Agents use the "bpm1.(eu).laserfiche.com/ca" endpoints. These have DNS A records with multiple entries that resolve to pairs or trios of AWS load balancers that route traffic to backend services. See for example MXToolbox DNS Lookup "bpm1.laserfiche.ca".

AWS' documentation states:

The IP addresses for [AWS-managed Load Balancers] change over time. Don't use this information to statically configure your applications to point to these IP addresses.

My understanding is that these IPs can change as AWS swaps out the underlying instances for maintenance, etc. In short, not something we control with the current architecture, and we can't speak for AWS on how often the IPs may change. AWS owns on the order of a hundred million IPv4 addresses.

The DNS records for "bpm1.*" have a TTL of 60s, so it would be reasonable to run a PowerShell firewall rule updater script at that frequency.

Regarding this error message:

Client with IP address '182.18.213.183' is not allowed to access the server. To enable access, use the Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range.

I'm not sure where this message originated from, or what the client and server are in this context. Is that client IP address a public IP for the Azure VM hosting Remote Agent?

0 0
replied on October 31, 2023

Samuel,

Thank you for the suggestions above and for continuing to discuss internally. 

The fact that not only are the IP addresses out of range, but grossly out of range creates significant issues for us.  These connections seem awfully fragile.  What can be done to stabilize the IP addresses, if not a fixed IP, but at least to the IPs in the documentation?  

Thank you

Franz

0 0
replied on October 31, 2023 • Show version history

Franz,

Please see the updates to my original reply and this new comment. To summarize:

The documented "Laserfiche Cloud IP Addresses" are specifically only for outbound traffic originating from Laserfiche Cloud for Web Request Rules and Application Connections and are completely unrelated to any Remote Agent functionality.

As Remote Agents connect to "bpm1.*" endpoints that resolve to AWS load balancers with AWS-managed IPs, those can span the entirety of the AWS IP address ranges for the EC2 service in the us-west-2 (USA), ca-central-1 (Canada), or eu-west-1 (EU) regions. Note that AWS regularly buys more IPv4 space and adds it to this pool. See: AWS and their Billions in IPv4 addresses

If you want to cover all the possible CIDR ranges, you can programmatically ingest the json file AWS provides and pull all the "ip_prefix" values where:

      "region": "us-west-2", (or ca-central-1 or eu-west-1)
      "service": "EC2",

While there are AWS architectures that can provide fixed static IPs, those aren't what our frontend uses today and changing would be a non-trivial technical effort. Though we're aware of the challenges dynamic IPs can present in these scenarios, those architectural changes aren't something that's likely to happen in the near future given (many) higher priorities.

0 0
replied on October 31, 2023

Re:

Regarding this error message:

Client with IP address '182.18.213.183' is not allowed to access the server. To enable access, use the Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range.

I'm not sure where this message originated from, or what the client and server are in this context. Is that client IP address a public IP for the Azure VM hosting Remote Agent?

 

This is the message from inside the Laserfiche business process monitor, history.  My understanding is the IP address listed there is the one LF is attempting to connect to our remote agent from and if we allowlist that IP on the firewall, then the BP starts to work again until the next time it changes. 

Note the IP address is currently 182.18.213.183 and that is not matching the domain name you've given that is bpm1.laserfiche.ca (15.223.140.212) so even if we were to write a PS script, how can it know how to obtain the 182.* ip address?

Can i also say, the remote agent has been working solidly for months but why has the IP address suddenly started to change now?

 

0 0
replied on October 31, 2023

and ip-ranges.amazonaws.com/ip-ranges.json does not list 182.18.213.183 so we cannot ingest it to dynamically configure the firewall as you've suggested. It seems that unless we can allow list the entire IPv4 block we've got a big problem with remote agent in this scenario, it can never be reliable like this.

0 0
replied on November 1, 2023

Our job failed again today, the message inside the workflow is "Cannot open server '****' requested by the login. Client with IP address '120.28.227.141' is not allowed to access the server.  To enable access, use the Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range.  It may take up to five minutes for this change to take effect.

 

Since the request can come from any of 4 billion IP addresses, what exactly can we do about this please? Must we insert a firewall rule to allow traffic from 1.0.0.0 to 255.255.255.255? 

0 0
replied on November 1, 2023 • Show version history

Sean, first, please reply to comments rather than the post so they thread. This is becoming increasingly difficult to follow with all the top level replies.

The error message you're receiving there is not about core Remote Agent to Laserfiche Cloud communication. As I've described, Remote Agent uses an outbound-only connectivity model where it initiates a TCP connection to the bpm1.laserfiche.ca endpoint, which is then kept open so the Remote Agent can receive tasks. Laserfiche Cloud does not initiate connections to Remote Agents. Remote Agent queries against external data sources come from the Remote Agent (that you host on your VM), not Laserfiche Cloud.

Accordingly, this statement is not correct/applicable:

Since the request can come from any of 4 billion IP addresses, what exactly can we do about this please? Must we insert a firewall rule to allow traffic from 1.0.0.0 to 255.255.255.255? 

Please read the error message closely. The "Client" cannot be Laserfiche Cloud, because Laserfiche Cloud does not initiate connections to Remote Agent. Neither Remote Agent nor Laserfiche Cloud have any concept of the Azure Management Portal. That tells me the error message being reported in Laserfiche is one that it received from somewhere else and is passing on. The question is from where?

Googling the error message returns dozen of results indicating that error message comes from Azure SQL Database in scenarios where its IP-based firewall feature is enabled. You mentioned that the query "returns data from an azure sql server on a VM", but I cannot find any indication that SQL Server on Azure VMs can return that error code. Every mention of it I've found is for Azure SQL Database.

Examples:

The "Client with IP address xxx" here appears to for your Remote Agent server that is executing a Query Rule against an Azure SQL Database registered as an External Data Source. Since all the IP addresses in those error messages have been public IPv4 addresses, either the Remote Agent VM has a public interface that's changing or the traffic is going through outbound NAT with changing addresses. The mention of public IP addresses suggests the query is going to a public endpoint for the database. Consider setting up Private Link to establish private/internal network connectivity to the Azure SQL Database instance so everything between Remote Agent and SQL uses private IPv4 addresses in subnets that you specify.

Assuming this is in fact an Azure SQL Database as the error message suggests, your next step should be to review: 

Then consult with other people/teams at your organization as applicable and decide how you want to proceed.

The Remote Agent is receiving this error from SQL when it attempts to connect, and is surfacing that error in the logs. No Laserfiche Cloud IPs are attempting to connect directly to the SQL instance. The "changing IPs" in this scenario are not Laserfiche Cloud's.

1 0
You are not allowed to follow up in this post.

Sign in to reply to this post.

Details
Related Posts
Loading ...