Swim faster?
Only half joking... if its a sports pool (which are kept at a lower temp - swimming in a warm pool when you're sweating is grim), you might really benefit from increasing your swimming speed or learning a new, more intense stroke. Breaststroke and head above water is relaxing but not very energetically intense as you don't involve your back that much. You could buy some goggles and do the olympic style breaststroke instead. I don't know which level of swimming you're at, maybe getting a coach for a session or two would also be worth it.
Otherwise, do some warmup exercises to get you sweating, then you'll be glad of the pool to cool you down. Planks are a good one, pushups, jumping jacks, ... Could you walk there instead of driving? Or take the bicycle?