I polished all the contact points inside, and it did help. Not as smooth as a unidirectional mag release, but a god start. I put a little Tetra grease on the V block too.
I will try pushing up on the mag.
As to aftermarket fixes, I did see that APEX makes a replacement mag release that is supposed to address this issue. It makes the release unidirectional instead of ambi. but this doesn't bother me at all. It's $68? I believe, which is a bit spendy.