This isn't really a Laserfiche question, so you'll probably find better information elsewhere.  My guess is that you are seeing one of the following, depending on the specifics of your project:

1. IIS is loading everything in the bin directory, including something that has that as a dependency.  http://stackoverflow.com/questions/508034/how-to-stop-asp-net-from-loading-all-assemblies-from-bin-on-first-load
2. You have an explicit reference to the dll in your web.config, possibly added automatically by Visual Studio.
3. What constitutes a dependency in .Net can be complicated.  Even if no thread would execute code in the LF SDK, all types present in a class or function have to be resolved.

For the last point, consider a function like:

void GetData(bool fromLaserfiche) {
    if (fromLaserfiche) {
        DocumentInfo doc = Document.GetInfo(Id, GetSession());
    }
    else {
        Stream s = GetDataStream();
    }
}

Because there are SDK types used in this function, the SDK has to be loaded in before any code here is executed.  In particular, it doesn't matter what the value you pass in for `fromLaserfiche`.