May Allah bless you for your firmness in your belief.

What you are doing is 100% correct and you shouldn’t listen to him. You shouldn’t also wear perfume when he has guests in the home who will find your scent.

My husband likes perfume alot and insists that I must wear it when we are going out together and that it is not haram if other men smell it as long as we are together I refuse and this is causing us to fight. What if I wear it at home and i have to serve