I ended up making an angle adjustable bracket for the pointer with my other toy, 3d printer.
The original bracket came with my machine is on the right. My mod also comes with a little knot for a focus gauge

My red dot pointer is now pointing at exactly where it should be.
Did anyone else have the same issue? Is there any easier/better way to sort?
If so, I'd love to know.