I was talked to once about my practice of catch and release.
I pointed out to the brother that it was a conservation effort. That I caught so many fish, it would be reprehensible, and in some cases illegal, if I were to keep everything.
After that, I was never hassled about it again.
I am a fly fisherman as well.