The Job story is a very old one, called "Oriental" by many scholars because it probably was around long before the Israelites and their adopted God were in existence.
It was picked up as good yarn and adapted by the Priests who compiled the O.T. as we call it, after the Babylonian Exile.
We have to think of how women were looked upon and treated at the time the story originated, even if Job was referring to an actual sexual liaison that was not between husband and wife, at this time it would have been seen only as a Crime against the Property of another. Coveting what was not yours was seen as wrong at this time, Sexual acts, even in the mind of the beholder, did not become "Sin" until much much later.