The usual historical explanation is straightforward. The oldest view is that of the Yahwist/Elohist law code (conventionally dated to before the eighth century BC) which states: "I, Yahweh your God, am a jealous God, punishing the children for the sin of the fathers to the third and fourth generation of those who hate me" (Exodus 20:5). This stance on jurisprudence was common throughout the ANE (cf. Herodotus, Historia 3.119, Esther 9:13-14, Daniel 6:25), and reflects the cultural legacy that Israel inherited from her neighbors. Stories from the time that reflect this theological view can be found in 2 Samuel 21:1-9 concerning the sons of Saul and 1 Kings 21:20-21, 2 Kings 10:1-11 concerning the sons of Ahab. The story of Yahweh taking the life of the child of David and Bathsheba (born in the context of adultery and murder) is another well-known example (2 Samuel 12:14-15).
The legal reform under King Josiah (seventh century BC), which produced the Deuteronomistic code, emphasized individual responsibility, and thus set aside the earlier stipulations: "Fathers may not be put to death for their sons, nor sons for fathers. Each is to be put to death for his own sin" (Deuteronomy 24:16). However portions of the old law-code were incorporated into Deuteronomy nonetheless with some tension between the two (cf. 5:9, which allows Yahweh to reserve the right to exact vengeance). The Deuteronomistic perspective is reflected in later stories in the History, such as 2 Kings 14:6. This doctrine on individual responsibility is adopted by the priestly class in the exilic period (sixth century BC), cf. Ezekiel who was adamant that Yahweh "will no longer require" punishment on children for their fathers' sins (ch. 18), even making the compassionate Yahweh declare: "I take no pleasure in the death of anyone" (18:21). This is a considerable theological development from the "jealous god" of Exodus 20:5. The theme of Yahweh's compassion was carried even further in the post-exilic period; the book of Jonah (fifth-fourth century BC) in particular is notable for portraying Yahweh as repenting from his threat on destroying a Gentile city, showing compassion on little children ("who cannot tell their right hand from their left") and even the animals of the city (Jonah 3:7, 4:11).
...OUTLAW