If you’ve had a discussion about transgender rights with someone who isn’t convinced, you’ve probably encountered an argument along the lines of “it’s just weird”. Despite this argument’s obvious weakness, it seems to be a sticking point with enough people that it’s worth addressing. In my experience, this attitude stems from a misunderstanding of categories: both how they’re useful and how they can fail. But first, a whale metaphor!
The blogger Scott Alexander offered the following example in his post, “The Categories Were Made For Man, Not Man For The Categories”: genetically, whales are within the category of mammals. However, if you consider how an ancient civilization might regard whales, they’re likely to see them as big fish. After all, you hunt mammals on horseback and you hunt fish from boats. No matter how much you explain to ancient people that, no, really, whales are in the same category as deer and cows, it won’t matter. 
This is because most members of categories are clearly defined. Deer are warm-blooded, have live young, are genetically close cows and are best hunted from horseback. Fish are cold-blooded, have eggs, are genetically close to sturgeon and are hunted from boats.
Because genetics are important to biologists, to whom we give the power to decide edge cases in the 21^st century, we let those genetics decide edge cases like whales and platypuses. However, in the way-way-back times when hunters decided the edge cases, whales were categorized as fish.
The question of whether whales really are mammals isn’t even meaningful. The word “mammal” is just shorthand for a bundle of attributes where most mammals are consistent but any given mammal might deviate. In cases of deviation, whether the difference is sufficient to put them in a new category depends on what you’re using categories for. 
If you’re doing biology, you should treat whales like you treat animals that are biologically similar to whales. If you’re hunting, you should treat whales like things which are hunted in a manner similar to whales.
Like whether a given animal is a mammal or a fish, gender is shorthand for a collection of attributes where people are broadly consistent. A non-exhaustive list might include XX or XY chromosomes, testosterone-estrogen hormone balance, external appearance and self-identification.
Just like with mammals, there are edge cases. Someone with congenital androgen insensitivity might have XY chromosomes and more testosterone than estrogen but might look like a woman and think of themselves as a woman. A transgender individual is simply someone whose self-identification goes one way while their genetics, hormones and possibly their appearance go the other. Given that these edge cases exist, how we decide them depends on what we’re doing.
If there was a hypothetical drug which acted on DNA to cure the common cold in women but makes men violently ill, a trans woman who still had XY chromosomes should absolutely not take the drug. This is because, in this instance, “are you a man or woman” is really asking “do you have an XX chromosome or an XY chromosome”. 
Likewise, a drug which acts on hormones should be taken or avoided by people based on their hormone balance. So a drug which cures people with high estrogen while sickening people with high testosterone should be avoided by trans men who haven’t had hormone reassignment, because the relevant factor in gender categorizations is, in this instance, hormone balance. 
The reason I used medical examples for the above two are because, outside of medical contexts, genes and hormones aren’t usually relevant in gender categorizations. Far more important are appearance and self-identification.
For most of history, appearance has been the accepted tiebreaker for edge cases in gender. After all, genetics and hormones are fairly recent discoveries. In the past, preferences about gender were far less important that whether you could plow a field. The transgender rights movement can be largely seen as an attempt to shift the typical tiebreaker from appearance to self-identification.
This is entirely appropriate. Since it is usually considered inappropriate in typical social interaction to intentionally make one of the participants uncomfortable, it is preferable to use self-identification as the tiebreaker in social situations.
The hypothetical conservative from earlier might protest. They might think that propriety dictates that they should refer to trans people as their preferred gender but they’re really whatever gender they were assigned. But, as with the mammal/fish dichotomy, “are you a man or a woman” isn’t meaningful in itself but is, instead, a substitute for an array of questions about the typical indicators of gender.
Therefore, saying “I’ll call a trans woman ‘she’ when talking about her but she’s really a man” is borderline incoherent because “man” and “woman” are signifiers —  not meaningful in themselves but only as indications of the determinants of gender.
Categories are how our brains are wired to sort things. In the ancient world, edge cases were few and far between enough that it was better to save processor power on judgments by putting things into categories and then reacting to those categories, instead of the thing itself.
But we’re not in the ancient world anymore. To function, we need to notice how our brains are working and correct them when they misbehave. Caitlin Jenner is a woman. Also, you can’t hunt whales with a horse. I tried. It didn’t end well.