He sounds like a classic narcissist, manipulative, charming, intelligent, lack of empathy...
Yes you might think that he’s empathetic, because he’s sending you flowers and sending you messages of support, but he doesn’t mean any of it, it’s all manipulation. He is doing these things to make himself appear charming and like he’s the good one in all of this. He’s clever, that’s why he’s doing it, he’s not doing any of it because he actually cares, it’s all a tactic.
Whatever you did years ago can’t have been that bad, I don’t think he ever had any intentions of having a proper relationship with you, he just used a mistake that made as his excuse to get out before you got committed so that he could manipulate you into being his bit on the side. He made himself into the victim so that you would feel guilt for the relationship going wrong, even though it was him that twisted it that way.
the line “don’t you know how hard it is for me” screams red flags of narcissism and manipulation. This isn’t something that most normal people say, he’s making himself into the victim so that you will feel guilty and then he will have you where he wants you. if it was that hard for him he would have left his girlfriend, simple as.
Cut all ties with him, please, you don’t deserve this. I’d also be very interested to see his reaction once he does know his grip is slipping.