That is all so true. I also think motivation has a lot to do with it too. My wife straight up dresses for me because she loves me and knows I like it. Since she is not dressing for others attention, I think its a look she can pull off.
But people also suck.
My wife and I are probably some of the most humble unassuming people in the world, but we do like to dress up and most of the time are some of the best dressed people at a venue. Not over the top bling-bling, but sharply dressed. Yet we hear snide comments a lot. Most of the time it is my wife who hears them from other women in the bathroom, or walking past just loud enough to her husband but so that we can hear, and occasion for us as a couple.
I think it just becomes unusual as more and more people dress down, and in doing so they either feel guilty about how they constantly dress, or flat out make the wrong conclusions on why we do dress nice. For us, its all about this: "You feel how you are dressed". Spend a week in the wilderness bush crafting it, you are going to feel gross and yucky. But dress nicely, and you feel better about yourself.