“That’s When” is a duet describing the aftermath of a relationship breakup, with Swift in the role of the individual who stepped away from a relationship due to needing space and Keith Urban contemplating whether he should take her back. It was originally written for Swift’s sophomore album, Fearless, but was cut. In April 2021 it was one of the “From the Vault” tracks on Swift’s re-release of Fearless.

The demo version was sung only by Swift, with notably different lyrics, and Swift in the position of needing to decide whether to take back the other person in the relationship. Swift has previously performed with Urban and opened for him on tour, and he covered her 2019 ballad “Lover”.