It's not surprising, at least to me, that this topic has generated almost as much heat as light. But may I add a bit of linguistics (light) to the conversation?
There is nothing inappropriate about the word niggardly. Perhaps if you deliberately misspelled it with an e instead of an a, but in conversation it is just a descriptive word. It means cheap or miserly and has no connection whatsoever with a bigotted take on skin colour. The two words derived independently but they have no doubt affected each others interpretation over time. Niggardly, like faggot, is still appropriately used in some segments of modern, English speaking society, but only by elderly speakers. It is being drummed out of the vernacular because it sounds like something else. The differences with the word faggot are two. One is that it has a couple of meanings, one derogatory and the other (cigarette) not but the main difference is it is slang. Whether the word is offensive, or taken by the hearer as offensive, should depend on context, ignorance of the genesis of the English language notwithstanding.
Another example, use of the slang word "fanny" may be considered completely appropriate in the United States but can be very offensive in the United Kingdom, not because the British are prudes and the Americans are boorish but because the meaning of the word is quite different in each of the countries.
The point I am trying to make is the effect of using the word cult is entirely dependent on the sensitivities of the listener.
Connotations of the word cult range from an enthusiastic following to something dark and evil. Use the word in conversation with someone who will without question take the latter connotation to describe what that someone sincerely believes in, like a Jehovah's Witness, and you will have uttered a deeply offensive slur. You will have closed the door on the conversation, maybe never to open it again.
So, go ahead, call a spade a spade, but don't expect roses in return.