Parented Objects Position Offset

I'm having an issue parenting an object correctly over photon. While I can successfully parent the object via an RPC, the object is offset from the correct position by a large amount on the remote clients.

My scenario is a client picks up an item (with a photon view) which is then parented to their hand locally. The remote clients have an object which tracks the position of the first players hand (via a photon view) and the item. When the item has been picked up, the remote players receive an RPC telling them to parent the item to the object that tracks the first players hand. This is successful and the item is now parented to the object tracking the first players hand. However it is also offset by a large amount.

I've tried setting the local position and rotation to 0 but the value keeps getting changed back. I suspect that the items photon view is interfering with the parents photonview and offsetting the position.

What could be causing this issue and what would be the correct way to resolve it? I have to do the parenting via RPC - if I don't, as soon as the object is parented to first players hand, the clients stop seeing the correct position.