ANSWER:

This is not shirk but shows your lack of iman. If they were slandering your mother or sister, would you have continued to sit there? 

If in a religious gathering people are discussing about shirk/kufr/bida’ah and I do not move out of the place thinking what the people would think of me if I leave or thinking that people would think bad of me then is this minor shirk and show off?