No one is saying that it was acceptable but it was regularly done years ago.
However, the argument would be better were it based on fact. They were not hidden away because of their connection to the royals, any more than their Trefusis cousins were. They were hidden away because that's unfortunately what happened in those days in many families. Three of their cousins also with a Trefusis mother, were also in full time care due to mental disability. It appears to have been commonly thought that the Trefusis family had a genetic problem rather than the Bowes-Lyons; Diana Spencer's father broke off his engagement to the then Lady Anne Coke (later Lady Anne Tennant, Lady Glenconner, lady-in-waiting to Princess Margaret) because of her Trefusis grandmother and their "bad blood". So nothing to do with the royals. Or was Rosemary Kennedy hidden away for a link to the royals too?!
Lady Elizabeth Shakerley, another Bowes-Lyon cousin, disputed the notion that the family had forgotten the two sisters, her aunts, saying that they were very much part of the family as sisters to her mother, Princess Anne of Denmark (but only the British royals get flak for it). It is possible that, like Rosemary Kennedy, they had lived as members of the family until their behaviour got too hard to deal with, and they were then institutionalised as was very common. Would they have recognised visitors by that stage? Possibly not. Rosemary Kennedy's parents did not visit her either and her siblings only found out her location in 1961.
So, sadly, this was not restricted to the royals but a relatively common way of dealing with mental disability during that era. Visits could distress both the patient and their relatives; particularly if the patient could no longer recognise them and the relatives were unable to cope with this and/or their behaviour. This is what eventually happened with my relative.