But what if I’m the one that holds and I always intend to drop

Then you're doing a command grab and the other person should have dodged it