Microsoft documentation makes it sound like you don't have to specify encryption in the repository creation wizard. If SQL is set to use force SSL, then the SQL Native Client (which LFS uses to connect) will use it. See this page in the SQL docs for configuration.

Alternatively, or if you don't want to force SSL in SQL, after creating the repository, you can add a string value called "ConnectionString" to the repository key under HKLM\Software\laserfiche\Engine\8.0\Repositories. The value should be the connection string including the "Encrypt=yes" flag. Something like this:

Driver={SQL Server Native Client 10.0};Server=myServerAddress;
Database=myDataBase;Trusted_Connection=yes;Encrypt=yes;

Then restart the server to make it use this connection string instead of the existing EncryptedConnectionString one (you can leave this one in, the server will prefer the unencrypted one if it exists).