But if you actually do walk away and do your own thing without them, they have to chase after you to make you feel bad, ruin whatever you're doing, and return things to the status quo.
It makes them feel weak and desperate, and they hate that.
Maybe they'll try to use those feelings of inferiority to try to guilt and neg you into coming back.
Maybe they'll talk about how both sides are wrong (but how you're more wrong for going away so you must definitely come back).