Open Mind, the only reason I would wait for the police would be to make public their stupidity. If you have been invited by an elder, have books on order at the literature desk, then it seems to me that the OP has some right to be there if he hasn't done anything wrong. And with the implied invitation(at the very LEAST) that he has, why would he not have a right to demand that the big unfamiliar jerk who is ordering him around explain to him who HE is and what right HE has to order him out. But its easy to have 'stones' when you are simply writing a post on JWD! :) I don't remember the OP's status, but it seems as if he was familiar enough to know that the guy ordering him around hadn't been there VERY long in any case. So he might be able to legitimately (because of that implied invite) ask those questions.
Arresting him for trespassing would be kind of ironic-a public place of (alleged) worship. . .and he was INVITED. And they fight all the way to the supremes for the right to invade our homes at will and ask us invasive questions. Who among us would not appreciate that police log write up in the local paper?