I don't really have any advice about the holiday, but I did want to share sympathy. My brother is similar. He too is home too much and is very isolated (he'd never agree though) and definitely doesn't get enough social/ general contact with other adults. Especially people not-of-his-choosing, yanno? He only sees people occasionally and then they're only people who aren't going to challenge him at all. He's completely lost the skill of biting his tongue/ nodding politely/ etc. He's become a bit controlling - not in ways you'd really notice, but he's used to running the house, kids, weekend plans, general family narrative/agenda etc, so when something pops up that changes that, he definitely does his best to adjust but I do think it throws him quietly.
TBH I think it's extra weird of your brother since he's not sharing a lodge with your son plus he has his own not insignificant family he'll be busy with - I wonder how he was expecting your 28 year old son to fit into his plans or his perception of how the holiday would go?? Was he expecting his nephew to be in the subtropical swimming paradise for hours with his 3?!