You miss the brotherhood. Nothing wrong with that.
While many on this board have very real horror stories about the people and elders they had to deal with, many of us actually enjoyed the people in our congregations and have nothing but fond memories in that regard.
My own bad memories of only a few people could legitimately be chalked up to personality differences that we were adult enough to navigate peacefully (with the exception of one elder.... but that's a topic for another thread).
I never felt particularly plugged in with the folks in my congregation. Not any meaningful, close friendships. Still, I felt the loss of brotherhood when I left. It's a sizable crater in your heart that will fill up with the passage of time and new friends.