How can spiritual still not be, in part, physical?
You're simply confusing yourself by trying to look at both definitions at the same time. "Spirituality" for an atheist has nothing to do with the spirit world, it's just another word for "awe". But religious people feel that they do have a part of them (placed there by God) that can sense a spirit realm. Is that part physical? That's not really relevant for them. Perhaps they believe it's their soul. Whatever it is, it either senses, or is composed of, something immaterial, and thus cannot be apparently tested for scientifically.
So one camp has their definition of "spirituality" as "knowing that there's something out there, above our own existence", and the other has their definition of "feeling awe or appreciation of nature and the universe". Personally I don't think non-religious people should ever use the word "spiritual", for precisely this reason, because it causes confusion, but there you have it.