By that logic, he should have just kept Dooley on, since he bears no responsibility for the performance of coaches he didn't hire. (actually, I wouldn't be surprised if that's the card he's been playing and that's why he really wanted to give Dooley a 4th year).
The fact of the matter is, it's been almost 3 years. There aren't many jobs you can hold for 3 years and show nothing but dismal results, simply because of your predecessor. And if the situation you took over was so bad, you should be able to show some improvement after 3 years, right? Not just the same results or worse.
Hart kept Dooley as long as he possibly could. He signed off on the Sunseri hire. He spent a good deal of money hiring the new coach (significantly more than SEC champion Auburn spent on Malzahn) and it has yet to yield positive on field results. During his tenure, basketball attendance and football attendance has dwindled. At what point will people decide to hold him accountable for anything?