As a parent it is up to you to protect your kids and set boundaries for others to respect.
I don’t think they go without Jackson because too many people ask for pics. If this was the case, you just keep walking and tell people you are there with your kid, sorry no pics.
I do believe they don’t bring him because they are selfish and want to do what they want. They want to have their fun and eat their food and they do not want to consider the needs of the children, because, well, that isn’t fun. This is true for everyday basic things, such as Target or going to a fast food place.
It sucks that they still do not consider the needs of their children at this point. Taking so many vacations mid-week is ridiculous. Go on the weekends! The excuse that it was already booked is bogus. They had time to reschedule.
The answer to why they do anything is because it suits them. They suck as parents.