He's right though, it's inefficient and an extreme inconvenience to a law-abiding citizen to be held for 4 hours without a real reason. The fact that he was trying to pick up his kids makes it worse. When they do this to someone, they should be required to repay them for the inconvenience.
If it ever happens again, you don't have to consent to searches if they ask. If they do it without permission, you can file a complaint for an unlawful search.
Cops actually have a right to search based on "reasonable suspicion", which is up to their judgement, so whenever they want.