I would just ask him "Is it important to you that I come see your talk?" if he says that it is then I would tell him "no problem I will be there for you." Go for his talk and leave immediately following it. I think he would get the point that you came for the sole purpose of seeing him give his talk, especially if you didn't bring a bible or song book or anything, or didn't even try to fit in a little bit.
If it were me I'd go to it. I almost went to my friend's first public talk because I supported him, they decided not to ask me because I'd be too distracting.