ANSWER:
You must not break relationships even because of this. You should try to change the topic and give them dawah that backbiting is a major sin but if they persist then you should leave the room. You must not be part of any haram activities with them.
If it’s totally unavoidable then you must at least maintain ties over the phone.