Your are missing the full view of rights here. You can marry anyone you want but what the fundamental idea (at least in the minds of many) of what marriage "means" has been changed.
Marriage is both a legal binding (fits your scenario) and also a symbolic binding. For 100's of years, there was a particular symbolism associated with this event. This is what I really mean by "social institution" and "man-made". It is a collective symbolism, ritual that has been accumulated over time.
If the courts mandate a change in this, they are taking away the rights of people to create, define and practice this particular symbolic institution.
As I've repeatedly stated, the legal aspects and rights are one issue. These can be covered via civil unions. The symbolic aspects are every bit as real but I believe we as individuals should be allowed to define, practice, and defend as we please.
Finally, I recognize that many don't value the symbolism to the same extent that others do. However, that doesn't justify a court solution to mandate change.