Associate, AI Model Validation, RQA

Overview

Associate on BlackRock's Risk & Quantitative Analysis (RQA) Enterprise Risk team, performing independent validation of AI tools and agents used across BlackRock, including agentic systems and RAG applications.

What you'll do

• Conduct independent validation of AI usage in workflows, including testing for accuracy, efficiency of guardrails, orchestration, memory management and token usage.
• Design and implement comprehensive validation strategies and test plans tailored to each AI use case.
• Define clear success criteria and quality benchmarks to measure whether the AI solution meets all requirements and standards.
• Understand tool architecture, potential vulnerabilities and weaknesses.
• Assess conceptual soundness, methodology, and alignment with intended business use.
• Review data quality, preprocessing steps and engineering decisions for AI use cases.
• Execute functional and performance tests.
• Evaluate performance metrics, robustness, and limitations under various scenarios.
• Document validation findings and provide recommendations.
• Stay current on emerging AI technologies, validation techniques, and regulatory developments.
• Communicate validation conclusions to relevant stakeholders, including escalating identified findings, risks or gaps to senior leadership and approval committees.
• Collaborate with AI Engineering, Data Science, and Risk teams to enhance AI transparency and trustworthiness.

What you'll need

• 3-7 years' work experience in the IT or financial industry.
• Degree or advanced degree (MS, PhD) in Computer Science, Data Science, Statistics, Applied Mathematics, or related field.
• Strong understanding of generative AI technologies, including large language models (LLMs), retrieval-augmented generation (RAG), agentic systems, prompt engineering, tool use, and modern AI application architectures.
• Experience with Python and common AI development libraries and frameworks (PyTorch, TensorFlow or similar).
• Excellent analytical, communication, and documentation skills.
• Ability to work independently and challenge stakeholders constructively.
• Experience with one or more of the following: validating usage of large language models (LLMs) in a business context; evaluation methodologies, benchmark development, ground-truth creation, and AI quality assessment; AI observability, monitoring, performance measurement, and operational controls; AI orchestration frameworks, agent architectures, prompt engineering, memory management, and context optimization; exposure to cloud-based AI platforms and scalable model deployment; MCP, APIs, connectors and integrations of AI solutions with enterprise systems and data sources; responsible AI practices, including fairness, explainability, bias assessment, transparency, and model governance; fine-tuning or model customization techniques; security testing and risk assessment of AI systems, including prompt injection, jailbreak, and data leakage testing.

Details

• Location: GGN office (Gurugram, India), onsite.
• Hybrid work model: at least 4 days in the office per week, with the flexibility to work from home 1 day a week; some business groups may require more time in the office.