They released a self-titled debut EP in 2001, but received little attention or fanfare. Following a change of lineup, they released their followup album, Tell All Your Friends produced by Sal Villanueva on March 26, 2002 on Victory Records.

TBS's cover of "Tell All Your Friends".It was recorded in New Jersey's Big Blue Meanie Recording Studios. The album gave the band a first taste of success through the single, "Cute Without the 'E' (Cut from the Team)". Since then, they have played with bands such as Brand New and The Used, as well as a short stint on the 2003 Vans Warped Tour. Another change in their lineup occurred when singer/guitarist John Nolan and bassist Shaun Cooper left the band. Nolan and Cooper now play together in another project, Straylight Run. Because of these departures, the band's future was in doubt for some time. However, with the additions of guitarist/vocalist Fred Mascherino and bassist Matt Rubano, the band would move on.

2004 proved to be successful for Taking Back Sunday, with the band opening for Blink-182 and being one of the main headliners for the Vans Warped Tour. In addition, their second album, titled Where You Want To Be was released on July 27, 2004 (also on Victory Records). This album had a different sound to their previous one. Propelled by the single "A Decade Under the Influence", Where You Want to Be went on to debut at the #3 spot on the Billboard 200 albums chart, with sales of around 163,000 copies. Taking Back Sunday received even more mainstream exposure by appearing (on the day of Where You Want to Be's release) on the late night talk shows Jimmy Kimmel Live and Loveline, and by contributing the song "This Photograph Is Proof (I Know You Know)" to the soundtrack for Spider-Man 2. They have also contributed the song "Your Own Disaster" to the Elektra soundtrack and "Error Operator" to the recently released Fantastic Four movie soundtrack.

TBS's cover of "Where You Want To Be".They are currently recording their third album in Los Angeles for Warner Bros. Records (My Chemical Romance, Linkin Park). Warner Bros. Records is owned by Warner Music, one of the "big four" major record labels.