It was at the CO visit, during the Elders meeting, that he would conduct the 'necessary' business to write in the books.
True we all agreed to return the book, but with all our notes, that part of the book belonged to us. I was always ready for a confrontation over returning it, but no one ever asked for it back.
Years later, an Elder, I ran into, brought up to me my keeping the book in a deragatory way while at a Flea market. I didn't even respond, wasn't worth it.
But, the book was eventually destroyed with all my other JW stuff when I felt so inclined.