A user with the Manage Entry Access privilege has the ability to browse all documents in order to set security on it. Make sure your test user doesn't have that privilege. Otherwise, removing bypass browse from the user and restricting the browse entry access right should suffice.