Adding to the mystery of whether the fruit was literal, I present Genesis 3:22.
In this verse we find that God has almost been outdone and is on edge about what Adam and Eve might do next. They have already become like God [these are God's words] just as the serpent had said. So God is concerned that they might next eat from the other tree, the tree of life, and live forever. God cannot have this to be the outcome. So they are banished from the garden.
Yeah, that sounds like a real event.