Probably because Holly doesn't have the clout. Pat could make it happen, and Holly ain't no Pat. That, and Tennessee's faded star probably didn't give them much leverage.
Even in the recent down years, UT still has excellent attendance...far better than most schools. Money shouldn't be an issue.
If there's an implication that UT is not a "prestige" game any more, then why do schools schedule unranked schools from weaker conferences? What did Kentucky's men have to gain from playing Evansville and Utah Valley aside from being embarrassed?
Has anyone suggested that the previous UT coach and athletic department attempted to schedule more ranked teams but were flat out turned down?
That leaves the other possible explanation that Tennessee still has enough reputation that many good teams, while not fearing us, still aren't all that eager to take us on. That, or the prior regime here had all the challenges wanted between the few noncon top teams and the top SEC teams.
Currently we have six ranked SEC schools, including three top ten plus #13. Only the PAC-12 has more highly-ranked teams, and no other conference currently has as many ranked teams.
Some of us remember the year that Baylor had an average team and the coach cancelled the game and the LVs scrambled to find a replacement. Related stories on UNC and Duke series. Remember the year-year Memphis series? LVs cancelled the Old Dominion series as they declined although the OldD fans would have preferred the 1 home/2 away schedule to continue. and recently the TV - conference teams challenges that frequently adds a game.
(legislatures in some states required the premier state university to play other state universities - La is one example)