OP , did you check the left side of the Hammer as to whether it's rubbing the frame? This can have a pretty good effect on the double action pull.
If it is rubbing, go to Gunsmithing section and click on Hammer rubbing left side of frame and follow instructions.
You mentioned that you " kind of polished the internals".
On my PO 7 and po9 I polished all metal-to-metal contact points, including pins and the inside of each component that the pin goes through.
I went with 15 pound Hammer Springs, that will reduce the pull. I have had no safety issues at all. my P09 w/rmr is my EDC.