As Gary Danielson likes to say, you cant hide your quarterback. This is where Harbaugh deserves criticism, in year three he should have a couple of legitimate options at QB.
Abso-freakin'--lutely. Losing every year to OSU will get him sacked sooner rather than later, unless he is winning every other game on the schedule, and he is not. And even that would be a problem.