Couldn't you just block them? If you don't wanna straight up block them, couldn't you just block and unblock them so that way you haven't blocked them but you've removed them as a follower?

I tried, but it does not remove them as a follower after I unblock them again.

I have. Thats why I made this tweet.