Moving left to right is typical. Assuming you're shooting off a quality rest to eliminate the shooter from the equation, I think I'd try a different scope, and shoot it without the moderator if that's what you mean by mod in your OP, before I condemned the rifle. How did this rifle shoot before recrowning?
You might want to have a gunsmith look at the chamber end to make sure there isn't something in the beginning of the rifling causing the issue. It also might be a good idea to slug the barrel to make sure the crown isn't looser than the bore.
If all that turns out OK, and nothing can be found 'wrong', I guess I'd pillar bed the rifle. There's a lot of discussion on RFC as to whether action only, or full length is better. Most 452's shoot better than yours out of the box. Most 455's respond well to full length pillar bedding.
Good luck.