The D.O.E was a man of his time and upbringing and life experience. I think people of much younger generations totally misunderstand his jokes and remarks, to call him a Racist is silly, he is a Royal, and all people who are not Royals are naturally thought of by them as being different.
But few if any of his remarks seem to me to be malicious, they are not the usual racist tropes, and he didn't preclude anyone from the benefit of his humour. He enjoyed letting a little of the real Phillip show at times, we who have been JW's know what it is like to have to pretend, to always say the right thing, and nothing even slightly risqué or whatever, we should sympathise with the fella, he had hard role in life.