You have to bind a valid X.509/TLS certificate to the LFFTS port (TCP 5053) on the server hosting LFFTS first. I'm guessing you already have your cert bound to 443, the default TLS port for most Laserfiche applications.

In PowerShell:

$certhash = 'certificateThumbprintNoSpaces'
$guid = '{'+ (New-Guid) + '}'
netsh http add sslcert ipport=0.0.0.0:5053 certhash=$certhash appid=$guid

If you're using an older Windows Server version (2012 R2 or lower I believe) that doesn't support the New-Guid commandlet, generate one from here with the Braces and Hyphens options selected: https://www.guidgenerator.com/

If you're using an AD Certificate Authority to provision the cert (like you should in most cases), bear in mind that even though it will (likely) auto-renew, it will not automatically rebind the new cert to the port on renewal and TLS communication will break until that's updated. Here's a PowerShell script you can run as a Windows Scheduled Task to rebind the latest available cert for a given Subject on port 5053. Note that it uses the netsh http "update" command which only works on Windows Server 2019 and above. On earlier versions you have to delete, then recreate the binding.

# Author: Samuel Carson
# Last updated: 2022-07-25

$logPath = 'C:\Scripts\Logs'
$logName = 'CertBindingLog.txt'
$logFullPath = Join-Path -Path $logPath -ChildPath $logName
$dateTime = (Get-Date -Format yyyy-MM-dd--HH-mm)
$logStartMsg = "`n`n[$dateTime]"

if(!(Test-Path -Path $logFullPath)) {
    New-Item -Path $logPath -Name $logName -ItemType "file" -Value '[Log file for certificate binding update script]' -Force
}
Start-Sleep -s 1
Add-Content -Path $logFullPath -Value $logStartMsg

# Gets latest cert and updates binding on port 5053 (Lasrefiche Full-Text Search Service)
Add-Content -Path $logFullPath -Value '[Port 5053]'
#$hostname = [System.Net.Dns]::GetHostByName($env:computerName).HostName
$subject = 'yourCertificateSubjectNameHere, e.g., lffts.example.com'
$certhash = (Get-ChildItem cert:\LocalMachine\My | Where-Object { $_.Subject -like "*$subject*" -and ((Get-Date) -lt $_.NotAfter) } | Select-Object -First 1).Thumbprint
$guid = '{'+ (New-Guid) + '}'
$netshMsg = (netsh http update sslcert ipport=0.0.0.0:5053 certhash=$certhash appid=$guid) 
$errorStringNoBinding = 'The system cannot find the file specified' #Indicates binding does not exist
if ($netshMsg[2] -match $errorStringNoBinding) { #Matching error message on third line
    $netshMsg = (netsh http add sslcert ipport=0.0.0.0:5053 certhash=$certhash appid=$guid)
    Add-Content -Path $logFullPath -Value 'Binding not found, adding'
} 
Add-Content -Path $logFullPath -Value $netshMsg
$netshMsg = ''

# Add break at end of log file entry
Add-Content -Path $logFullPath -Value '----------------------------------'

You can also enable TLS for LFFTS through the Search Engine Configuration Utility command line program. If you're not familiar or comfortable with using that I'd recommend enabling the option through the Attach Repository wizard.