With time you'll realize that the fear and panic is just in your head. Allow yourself to be free from the mental control that you were conditioned to have as a JW.
Make steps to be out in public and do the things that you enjoy. Avoid (shun) them if you see them at a distance. If the circumstances require a face-to-face encounter, repeat a stock answer ("We're doing great! Good to see you too!!") that you've memorized.
With time they'll avoid you in public!
ginger