You do need to be firm. Whether they are showing sincere loving interest or whether they are on a power trip "treat these two imposters just the same" and simply say as many times as you need:
"Thank you for your interest, brother - I have some personal matters that need all my attention right now and so I won't be able to join in/join up/do that/have a shephering visit/go on the ministry with you/ if anything changes I'll let you know."
They will let you go or may offer help - if so politely say
"like I said brother I have some matters to attend to over the coming weeks - if anything changes I will let you know."
If they really persist just put your hand up and say "I know where you are if I need you, gotta run!" and walk away.
Rinse and repeat as often as you need.