I think Saban knew that it was other stuff that Saban hated. Kiffin marched to his own beat and didn't fall in line.
I heard a story a while back about Kiffin being there, after they won the Sugar Bowl Saban, Kiffin met with Sexton and talked for like 45 minutes post-game, when Lane hit the locker room Saban told him to get dressed and he could shower when they got back to the hotel. Instead Kiffin stayed and showered Yada Yada and the bus ended up leaving like 2 hours late waiting on him. Stuff like that drove Saban crazy.