My question to you is, if the 50 that are your tax clients stoped using your services, would that cause a financial strain.
Certainly you could replace them with new clients but the transition time could be a tax on the pocketbook (pun intended)
I agree that you should say nothing. If they find out they will see you as happy and successful.