Staff Quality Engineer ( Python , AI )

Overview

Staff Quality Engineer to build and extend the test automation platform for Arctic Wolf's security operations pipeline, including AI/ML and GenAI application testing.

What you'll do

• Create and extend software frameworks and tools to run, monitor, and report on automated tests.
• Build and maintain automation frameworks and CI/CD pipelines.
• Review testing strategies, approaches, and test plans and provide insight to development teams.
• Architect, design and build software to automate a variety of testing methods (e.g. API validation, performance/load testing, fuzzing).
• Develop and maintain test frameworks for GenAI-based and AI/ML applications, including model validation, LLM prompt-response consistency, hallucination detection, and regression testing for AI inference APIs.
• Implement evaluation pipelines to test LLM accuracy, latency, bias, toxicity, and reliability.
• Work as part of a team of highly skilled software professionals.
• Continuously learn and expand your technical horizons.

What you'll need

• 8+ years of experience shipping quality software.
• Experience building and maintaining automation frameworks and CI/CD pipelines.
• Experience deploying services on cloud computing platforms (e.g. AWS, Azure, GCP).
• Expert use of common tools for executing functional, load and fuzz testing (e.g. Locust, CATS).
• Love for driving high software quality.
• Excitement to work at the intersection of AIML, Big Data and the Cybersecurity problem space.

Nice to have

• Experience working on distributed systems and microservices architecture.

Details

• Location: Bengaluru, India.
• Background checks are required for this position.
• Conduct duties in accordance with AWN's Information Security policies, standards, processes, and controls.