I would just ask for a new chair as your existing one is causing discomfort. If they say no, consider investing in one yourself. I bought a very reasonably-priced one recently which has a comfort rating of 12 hours (most are 3-4 hours; the one you're using currently is probably designed for 1-2 hours at best [it sounds like a meeting room chair]). So definitely check its comfort rating - and weight allowance too; if you are on the heavier side, make sure your chair is designed to support you. You can pick up great secondhand options at office clearance stores that will tick all these boxes. It really is an investment in your healthcare, as it's really hard to come back from the sorts of injuries that using poorly designed equipment cause. Before I invested in my new chair, I would get terrible cramping in my lower back and abdomen, neck, and headaches - which obviously stemmed from using a 3-4 hour comfort rated chair, up to 18 hours every day (despite stopping for breaks and carrying out stretching exercises).
I also have a foot rest that has raised bumps built into it, which are really massaging on my bare feet (I work from home); this is just one type available, but is really good at forcing me to sit up straight:
Plus, I recently bought an additional lumbar support for my car seat and office chair (I just take it out with me), which also forces me to sit correctly ... it's sort of like this; I bought it off Amazon for less than $10. (Just a note on the chair style here - do NOT get one with arms - they encourage you to use them, which adds to aches and pains). I used to have the same issues sitting my car for prolonged periods, as I did with my former chair; I've had no issues since using my lumbar support.
My other recommendation is that you look up 'office exercises stretches' - there's a wealth of information available, including some great diagrams that you could print out and pin to a wall at your desk (or keep under your keyboard). Set a timer on your phone or PC for regular intervals throughout the day, and force yourself to do them. Also stand up and stretch a bit every 20 or so minutes.
If you make these improvements, you'll experience improvements in no time.
Just to add - if working at a screen all day, it should ideally be large and widescreen - definitely not laptop-sized. The top of the screen should be at normal eye level for you - don't have it too high or too low. There are free guides online that explain how to get the perfect setting.
A note on the desk, too - it's not essential to have a height-adjustable desk. If you do have one, great; but the chair is the key piece of equipment here. Get that sorted and everything else will feel manageable. I do think the main issues in your current set-up are with the chair and not undertaking stretching exercises / taking breaks.
And one final thought - if I feel like I'm getting a stiff neck, I apply Vicks Vapo Rub balm! Sounds crazy, but it really does work at ironing out any niggles. I use it on the backs of my calves too, if they're feeling a bit tense.
)