Why does the man who marries a divorced woman commit adultery, if the last husband committed adultery by remarrying?

In the gospels, Jesus says whoever divorces his wife, except for sexual immorality, and marries another commits adultery, and whoever marries her who has been divorced commits adultery. Matthew 5:31-32: It was also said, ‘Whoever divorces...