Having affairs is, or used to be, the done thing in the aristocracy. It’s only the plebs who are big on fidelity. Aristos had to marry well, but didn’t necessarily marry the person they wanted to. Lots of sneaking down the back passages during hunting weekends. Sex is like riding a horse to them. Giddy up old girl.
I have no doubt Penny M was Prince Phillip’s long term mistress and there were others before her. And the Queen had her Porchy (Lord Porchester, Prince Andrew looks like him). Prince Phillip was a beautiful Alpha male and still had something about him into middle age. The Queen was a beautiful young woman but she looked a bit homely next to him, despite being the most famous woman in the world. The Queen turned a blind eye. Her and PP were a partnership, together a long time, could never divorce, they were entitled to a private life which may have included others. That would be seen as perfectly normal in aristo land/monarchy. Prince Charles said to Princess Diana, when she complained about Camilla, that all Prince of Wales had mistresses, why shouldn’t he? So, they see it differently.