He occasionally puts out albums, but I'm pretty sure he writes little or none of his verses.
He's put out an album every 2-4 years since No Way Out. That's good enough for me. All sorts of rappers use ghostwriters, by that logic, Dr. Dre isn't a rapper either (he also isn't by your "occasional" album logic).
Dre never was really a rapper. His albums serve more as a showcase of his skills as a producer, as well as promotion for the dozens of featured rappers.
"Ghostwriter" is really only applicable if they're secretive about it. Dre and Diddy never really set out to convince the world that they could be viable MCs. They just want something to put them in front of the music rather than constantly behind it. It's a way of building a brand.
I don't think being a producer first and foremost excludes them from being rappers. There's plenty of rapper/producers.