I have configured an AD Trustee and able to test this following group in the Trustee Search.
However I am not able to use this same group in the Assign Rights Dialog.
What I have noticed is the only options in Assign Rights are Laserfiche Users and Laserfiche Trusted Users any other AD Users or Groups to not seem to be available.