I totally agree with this and have thought that for a long time.
Unless you're also the GM, head coaching in the NFL is a much more narrowly-focused job. I don't know if it is necessarily any easier, but it's more focused. You basically camp out in a film room all week with your guys to strategize and go to practice. With the exception of a young QB, your players are more or less already developed. It's extremely stressful and of course entirely performance-based, but there's no recruiting, less politics, and it is 100% about tactical football. You already have finished products as players, so it's up to you to figure out how to use them the best.
College coaches have to put up with an incredible about of BS. "BS" being defined as stuff that is secondary to what goes on on the field. If you're at a big time job, your career will ultimately hinge on whether or not you're able to convince 17 and 18-year-old divas to come to your school, and you have to go on essentially a 24-7-365 barnstorming tour in order to get them. CFB is a coach-driven sport, so you're expected to be the public face of the program, if not the entire state, and need to have at least decent media skills (especially if you aren't winning quite enough). The University itself is a political environment (way more than any NFL team is) and you need to have allies in the administration and among the boosters (again, especially if you're not winning enough).
NFL coaching is more of a "pure" football job, and I see why many coaches would prefer it to college. I love college football more than I love the NFL, but if I was in the coaching business I'd probably prefer working in the NFL.