I'm having trouble seeing where the two of you actually disagree. You're both arguing for a middle approach, what the Greeks would call "moderation"; neither of you care about every detail about what others think, but both of you do track their possible responses in order to fit in at least some of the time in "low cost" situations. Both of you go your own way when the stakes seem higher to you. I think it's more the word-tags you tend to prefer when describing boors that divides you, which is to say, semantics (of a kind.)
I think we pretty much agree (and I thank you for your "moderation"), but I go further. There is a fairly strong correlation between the thoughts and actions of others, including ourselves.
I care what others think of me, in part, selfishly, because I care what they do with those thoughts. And yes, sometimes being loved and being feared, just as an example, can cause others to give us money or attention, thus two distant feelings can have the same consequences.