“Bye Bye Baby” was originally an album outtake from Taylor Swift’s second studio album Fearless, before being re-recorded for Fearless (Taylor’s Version). A picture of Taylor’s early concept tracklist for Fearless shows that the original title for this song was “The One Thing”. Multiple lyrics were changed from the demo to final production. It was likely taken off of the original tracklist because of its similarities to Michelle Branch’s “Goodbye To You”.