Please don't take your baby to the game. That has to be an absolutely miserable experience for someone who will have no clue what's going on. I'd say 5-6 is a great age for your first game. I remember going with my dad, him sitting me down, getting me a hot dog and a big ole coke. Good memories.
The people in front of us brought their infant to the game. The screams made it cry, the heat made it sweaty, and you could tell the kid was absolutely miserable, yet the parents thought it was sooooooooo cute to tote him there in his orange ut onesie. Don't be that guy.