I'm public sector and currently in a similar position, I'm 15 weeks and applying for jobs left right and centre within my org (
but this is because my manager is such a piece of work I've been off sick with work related stress because her behaviour has massively escalated recently and I'm now having to do informal HR complaints and the like so I just have to get out for my own health lol). I do feel a certain level of awkwardness/cheekiness about it, but at the end of the day, you've got to do what's best for you. And when I told a senior I work with how I felt cheeky applying for a job she said "but you shouldn't feel bad, it's the law that you're not at a disadvantage" so I'm just going with that
I personally wouldn't withdraw my application now if I were you. Assuming your notice period is 4 weeks (?) if you were successful and offered the role and you start the role in 8 weeks, you'd only be 12 weeks pregnant, that's still five months or so in post!
That said, if you think the stress of changing job and having to learn a new role mid pregnancy is not worth any benefit from a new post, then yeah sure, but I wouldn't disadvantage yourself now just because of feeling awkward about it!