You probably already know much of this. Nonetheless, maybe the first two sections of this post may be helpful? https://eugeneyan.com/writing/content-moderation/

Content Moderation & Fraud Detection - Patterns in Industry

Collecting ground truth, data augmentation, cascading heuristics and models, and more.