CoCo,
One thing I really hate is when someone like that puts you on the spot in a way that they demand to know your reasons why.
I feel that I should not have to give an excuse or reason.
If I said "I can't or I won't" that should be sufficient.
But it is just more of an energy drain when someone expects that they have the right to approve whether your "no" is justified to them.
That irks me.
In a case like that I might have to say..."My t-shirt just expresses my feeling today." And I would let it go.
Sometimes when met with insistence, I might say "Are you trying to take advantage of my good nature?"
I have found that when some people are very selfish and needy, whatever time you give them it is still never enough, and comments like above just go in one ear and out the other.
With people like that, sometimes you have to get firm or a bit sassy to get your point across so they lay off for awhile.