I am a huge fan of taking the shuttle, but if you insist on hanging out downtown, I won't try to talk you into it. You've been here, so I guess you are prepared for the long walks. If it was me, I'd hang out downtown on Friday instead.
The Downtown Grill & Brewery has gone downhill, IMO. If you like the idea of a brewpub, Balter Beerworks has much better food (but not as conveniently located). At the moment, Vida is my favorite downtown restaurant. Stock & Barrel is awesome if you want a burger. Harvest is usually excellent, and they opened a location on Gay Street pretty recently. Barley's in the Old City is good for pizza and beer. Kaizen (Old City) is very good if you want Japanese. If you don't already have a reservation, you will have a very hard time getting in anywhere downtown Friday or Saturday. A little bit north of downtown you can find some cool places like The Oak Room and Central Flats and Taps.
Calhoun's on the River is great for game day, but will be packed. As has been mentioned, there is a Calhoun's a little west of Cedar Bluff. Also one a little farther west in Turkey Creek. However, there is much better food to be found, and you can actually get Calhoun's in the stadium (on the west concourse). Archer's has my favorite bbq in town, and they have locations all over the place. In west Knoxville, I like Season's and Connor's (both in Turkey Creek). Abridged on Parkside Drive has awesome burgers and is a great place to hang out.