1. Cost. Every PMC that has ever been hired has come with an inordinate cost associated with that contract. Are they "better" than using US troops? Nope. Many times, especially with the restrictions placed on them, they go out and hire third country nationals to perform the "base level" security while pocketing the rest of the funds.
2. They'll still be subsidized through the government in some shape or form. I take the airlines as an example of this. For every new airliner the big airlines purchase, there's generally a loan from the US government helping with that purchase cost. So, the government will still end up helping purchase higher end equipment for said contractors before it's said and done.
3. Regardless of how "private" it is, the deeds and actions of said contractor will still be linked and attributed to the US Government. And if they are a little more fast and loose with the rules of engagement and cause yet another Nisour Square massacre, those actions will be on the government's head. And you end up with stupid crap like this:
Emails Reveal Discord Over Blackwater Charges
Can't get anything else to fly? Toss in a weapons charge even though they were not on US soil and the weapons didn't fall under the purview of the USG at the time.
4. Even as much as you and I can complain about it, the professional aspect of the US Armed Forces cannot be overlooked. Sure, there might be some companies with a vested interest in what they are doing, but by and large, their overall goal is "get home alive and get paid." As you mentioned, the UCMJ aspect keeps most military in check from questionable actions. Not as much with PMC groups.
5. Before Huff jumps in with his "letters of marque and reprisal" argument, that's basically saying the US Government supports piracy and "terrorism". I use the terrorism term loosely since one man's freedom fighter is another man's terrorist. However, those letters of marque and reprisal basically give a carte blanche to groups to act in their own interests. Including actions which would seem extremely contrary to our government's beliefs.
No, we have a professional military for a reason. I'm all about stopping the wasteful spending we often see and providing better combat training, better equipment and better benefits to our troops rather than have to rely on third party contractors. Enough of the touchy feely social experiments and get back to the main mission of killing people and breaking their toys.