It's that last bit: people care more about their predetermined allegiance than policy.

People project their policy beliefs onto the party they support. That's part of why Dobbs is such a disaster for Republicans. A lot* of lifelong Republicans were convinced that their party wouldn't actually ban abortion. *if they lose 5% of their support, they're dead as a political party.