Oh, does blocking remove them from your followers now? Probably explains the low following count

Blocking should remove them from followers. But if you unblock them, they will follow again.