I can easily picture myself being like that elder. I have a natural tendency to be forgetful about a lot of things, even in my own life. Also, while I was one I was dealing with the abuse of my own child.
Elders can't win. Some really are no good but many are. If he remembered, there would be complaints that they never forget what someone has done wrong. If he forgets, then he just doesn't care.
Very much like threads about visiting disfellowshipped. If they come, they're bad because they won't leave you alone. If they don't come it proves that they don't care.