I've left my kid(s) twice to go abroad. My first holiday with my now husband was a week in Europe when my son was 4, we hadn't been together long enough for a "family" holiday, however we lived with my parents so they encouraged it as he would be at home the whole time.
The second time was a surprise city break honeymoon where my kids, aged 2 and 9, stayed with grandparents for a few days.
I'd say these were very one off circumstances and as a rule we would never go abroad to somewhere luxurious and only treat half of our kids to a sun £9.50 caravan holiday.