I think a lot of it can be attributed to slide weight. Out of the box, my 07 was much softer shooting than my P-01. Both guns weigh nearly the same, but the 07 slide is considerably heavier. I haven't fired a Phantom, but the slide looks to be lighter than the std SP-01. I'm probably alone in this opinion, but I found the 07 to be a better handling package than my 09(sold).
On a similar note, my brother in law has a P320 Compact Carry and a P320 X-Carry. The X-Carry has a lightened slide and is considerably more snappy compared to the standard compact.