“All These Things That I’ve Done” was written by frontman Brandon Flowers and features gospel choir The Sweet Inspirations. It was released as the third single from the band’s debut studio album Hot Fuss in 2004 in the United Kingdom, and as the fourth single in the United States.

The song is about Flowers' struggle with his Mormon faith. He hasn’t always strictly followed his religion, and he’s unsure whether God will still accept him. By the end, he’s decided to make his own way; he still believes (“I’ve got soul”) but he will never be completely devout, he won’t fight and die for his faith, he won’t try to push it on anyone else (“I’m not a soldier”).

The song peaked at #74 on the Billboard Hot 100 and #18 in the UK Singles Chart.