I wouldn't worry about it, I promise it's not about not having empathy! It's really hard to show genuine empathy when you get talked to a certain way, and a lot of the time it honestly makes me feel kinda gross.