Pretty quick is how it goes. Remove action/barrel from the stock, loosen the two grub screws securing the barrel to the action, slip the barrel out, slip the new barrel in, tighten the two grub screws to 35 in. lbs., mount the action/barrel back into the stock. Could be done in 5 minutes if you took time to have a drink or two of coffee during the process.
Proper inletting is one of the keys to accuracy with the 455 platform. My Varmint would shoot two bug holes, one at point of aim and the other 1/2" away at about 10:30 (315 degrees), with shots alternating between these two "nodes". Proper installation of pillars and glass bedding the action should solve an issue like this but for a variety of reasons I never got around to doing so. Ended up buying a McRee Precision G10 chassis and the results have been VERY encouraging. One still has to decide whether adding a $400+ stock/chassis to a $400 gun is "worth it" and the right answer for me might not be the right answer for you.