I went through this same thing I didnt post much on here about my confusion I just posted about my outing and how they went.
I started like you (and many others I'm sure) years ago going out at night hoping no one would be around. And then one day I was driving somewhere in heels and I stopped at a store to get something and I kinda forgot I was wearing my heels. Withing the first 30 feet of walking into the store a younger woman complimented me on my heels. I was completely thrown off guard and I looked down and realized I was still wearing them. Told her thank you and quickly finished my shopping and left.
Later to realize how much it doesnt matter that I was wearing heels. After that I started doing all my shopping in heels. Within a few months I was wearing skirt and pantyhose with mens shirts and jackets. To me they are way more fun than pants, more comfortable and alot of the times better looking!
The only part that has been a problem for me is my girlfriend hates all of it. Which I've been struggling to get across to her that they are just clothes and why cant I enjoy them. She thinking I must be gay because of it, but those of us here know that isnt the case..
Good luck to you and keep trying to get out and about! It makes everyday things more exciting and enjoyable!