The spririts don't come already inside... I bothered to look it up when bored once. Bad...yeah. Either way The board is just a medium to speak to the outside spirits. Spirits that may be around. That's why the rules say don't use it outside, more possibilities of "bad spirits," (Or so they say...good don't exist...). There's actually a lot of rules to using it that make it more than a game.
Don't use it outside.
If a spirit answers, don't let it knock the (what's that thing called?) off the board, if it knocks it off upside down, say goodbye, it's bad.
Always say goodbye at the end of every session.
And many others.
It's by far more than just a kids game.