Safeguards Enforcement Analyst, Violence & Extremism

Overview

As a Safeguards Enforcement Analyst focused on Violence & Extremism, you will build and run operational workflows to assess model behavior, make enforcement decisions, and develop evaluations across related policy areas.

What you'll do

• Design and architect automated enforcement systems and scalable review workflows with high accuracy.
• Develop and maintain evals to measure performance, surface regressions, and guide policy/model improvements.
• Partner with Engineering and Data Science to optimize detection and automated enforcement systems.
• Review flagged content to drive enforcement decisions and identify policy gaps, including novel misuse and emerging extremist tactics.
• Provide structured feedback to the Safeguards policy design team based on enforcement scenarios.
• Develop and maintain enforcement guidelines and reviewer documentation for consistent decisions.
• Identify and escalate emerging misuse patterns, novel attack vectors, and signs of coordinated violent extremist activity.

What you'll need

• Experience in policy enforcement, threat intelligence, counterterrorism, government, or a closely related field with direct exposure to harmful content and violent extremism.
• Experience standing up and scaling policy enforcement or content review workflows.
• Proficiency in SQL and/or other data analysis tools for insights from large datasets and monitoring workflow health.
• Ability to identify emerging risks and communicate findings to stakeholders including Product, Policy, Engineering, and Legal.
• Experience working with generative AI products, including writing effective prompts for content review and enforcement.
• Understanding of implementing product policies at scale, including in content moderation.