I've not swapped out just the barrel (replaced a couple of bad ones, but don't swap from .40 to 9 and back).
I have bought whole top ends (slide/barrel/etc.) to put on a different caliber frame. I felt the need for adding a new slide stop for use with the different caliber top end. Just to be sure the barrel always works/rubs/stresses/wears the slide stop shaft the same for that caliber. In my mind, switching back and forth on a single slide stop pin might result in increased wear/breakage.
Others might not have experienced that but I know you don't swap used parts around in an engine or swap new/used parts without making sure wear surfaces are as close to new as possible on the old parts before installing them with new parts (talking crank shafts/bearing, cams/lifters/push rods/rocker arms, etc.) I've seen the results of someone replacing a broken part with a new one without replacing the parts that worked with that new part.
Having said all that, I've thought about a conversion barrel in a couple pistols but never followed through with it.
Good luck with yours.