I think your doing great Flumps you are protecting your daughter and putting her welfare first (nevermind anyone else and their hurt feelings!).
That's their problem not your daughter's it sounds like your ex was abusive and unfortunately people like that seem to attract others who also do not have any boundaries (or self awareness).
It seems to me as if you have had to parent and be the strong one to a lot of people so please also make some time for yourself as well and don't put your needs last.
You are absolutely right though in being honest and refusing to whitewash things your sanity needs the truth! As do the people around you.
Instinctively you know that you need to keep your distance from people who bend and twist the truth if they cannot cope with reality it's not your problem or fault or your daughter's.
You are absolutely doing the right thing and I can only imagine how hard and tiring that must be at times.
Some people unfortunately are going to want to live in the past and constantly rehash it but you have already moved on I think because you have experienced the dark side of this man and are not in denial of it.
So you see the bigger picture you don't pine ove what could have been or what if because you know that really it was hell.
Some people will for their own reasons cling to their delusions as if their lives depended on it and then telling the truth can make you feel lonely or out of sorts!
I know that feeling unrelated but I went through that feeling when someone was abusive to me and yet when I told others no one wanted to believe me because he was a popular and charismatic figure!
At the end of the day though you have to be true to yourself and tell the truth (,difficult as it is surrounded by in some case enablers flying monkeys or even fellow sick people) misery loves company.
You closed that door once and you know you have the strength to do so again if needed.
Life can be beyond testing sometimes but you have already proved yourself again and again so take comfort from the fact that the more you do that (distance yourself use your instincts protect) the easier it gets and the better it will be for both you and your daughter in the long term.
So take care and take heart from the fact that this will get easier probably you are already nearly there (ironically that's when it hurts the most) and we are all here for you anytime you need us.