Bluesky doesn't have a way to remove followers quietly.. the only way to do it is to block them.. and if you unblock them, they're your follower again.