Grabbing a physbone and sticking it to the world

Hi, my avatar has some detachable parts which can be grabbed via physbone. If someone grabs it and poses it, though, it still translates along with the body part it was grabbed from. Is there any way to make it stick to the world coordinate space instead?

I tried using a VRCParentConstraint set to “freeze to world” but it seems that the physbone is overriding that.