I was told by an elder that it was not a disfellowshipping offence, that you just would not have any "privileges".
However, I would imagine that if one threw a party and invited others in the KH to it that could be construed as causing disunity. That would be different than just celebrating your own B-day or attending one.