I dislike bank holidays when I have to work, because people assume it's Sunday opening hours. So it's dead before ten, then dead again after 4, and it makes the day drag.
I also think a lot of people don't consider that for them to have an enjoyable bank holiday, someone else has to be working. Look what happened when the Queen died. People got excited about getting that extra day off, until they realised that shops, hospitality, and tourist attractions would also be closed. Suddenly the day off lost it's shine for a lot of people.