At age 22, Erika Wennerstrom moved to Cincinnati and met Mike Lamping. They hooked up and began playing gigs as the Heartless Bastards, which was originally formed as a four piece band with members Dave Colvin on drums and Michael Weinel. They played their first live show at The Comet, a bar in the Cincinnati community of Northside, in August 2003. After parting ways with Colvin and Weinel, the group reformed as a three piece with Kevin Vaughn on drums. The band was signed to Fat Possum Records in 2004 after Patrick Carney from The Black Keys passed on a demo he received from Erika.
Rolling Stone reviewed their debut album Stairs and Elevators and opined, “the Heartless Bastards are a small-town band who are ready to show the big city no mercy”. The Village Voice proclaimed: “deadeye accurate in pitch and message… what we’ve got is a hard, gnarled voice singing simple-seeming melodies that feel archetypal rather than ordinary, which is no easier to explain than it is to do”. Stylus gave the group high praise, stating, “(Erika Wennerstrom) and her two band mates have created an album with more rocking songs and fat hooks than most bands can dream of. It’s not just that they rock, it’s that you believe what you hear, that they love the sound they make, that Wennerstrom lays her soul bare in her lyrics without sounding like Sarah McLachlan, that the women of rock who labored to make it OK for a girl to dream of playing guitar deserve far better than Avril Lavigne or Kelly Osbourne as their descendants.”