Try to analyze why you feel guilty.
When you conclude (like you wrote) that you feel guilty because your children are sitting in the KH without you, consider this. Your children go to school without you and you probably will not feel guilty about that. Because you don't belong there at their school. The KH isn't your place neither. But you can support them, like you would do with school. You can think about bringing them, maybe regularly, maybe once in a while. You can even try to visit a meeting when they have parts. Not because you want to visit meetings, but because they are your fantastic family. You can prove them that their 'worldy' father still is a loving man, even if he has another opinion now.
There also is some nostalgia. Recognizable. How foolish we were back in those days, huh?
PSV - Milan : 1 - 1. Still a chance to go to the next round!