They fell off for two reasons. Their first EP and first LP were both amazing, even if they had different vocalists. Then they tried to write a lot more songs like You're Not Alone (or, possibly, I Never Wanted To, which Cove told me was his favourite of their songs) for their follow-up and it fell flat. The songwriting was mostly uninspired and the production was poor.
Secondly, Cove's voice dropped off. Some people didn't think he could sing live to start with but I saw them every time they toured Australia, on their own, with the Taste of Chaos tour (remember those?), with the Soundwave Tour, with Story of the Year. Cove sounded and performed great until that last tour, when he seemed uninterested and his vox were off. Shortly after he was dropped.
This band is one album (albeit quite a few years -- six, I think) removed from a killer record. I'm holiding onto some hope.
I just can't figure out why there was such a drastic drop in quality form their first to second album. Their S/T is amazing like you said. The guitar work and drumming is great, and every song is extremely catchy. ISOSG is the complete opposite. Weak guitar work, boring drumming, no catchiness at all. I can understand it not being as good as the S/T but it's just crazy how shitty they got.
I used to love Saosin, but with their parting ways with Cove being so cold, and the comments certain members have made about him, have left a bad taste in my mouth.
This pretty sums up my feelings. The band should have kept Cove IMO. While ISOSG was a big let down- it did have some solid jams and I'm sure with the right producer the band could have made a comeback on the follow up.