Software Engineer, Evals

Overview

Software Engineer, Evals on Glean's Evals & Observability team in Bangalore, building the measurement and quality systems that determine whether new models, prompts, retrieval strategies and agent workflows are ready to ship to enterprise customers.

What you'll do

• Design and build large-scale evaluation pipelines that measure assistant and agent quality across thousands of real user and synthetic workflows.
• Evaluate frontier model releases and the latest open-source model drops, building infrastructure and quality signals that surface regressions, tradeoffs and launch readiness.
• Build agent observability infrastructure, including trace enrichment, durable telemetry pipelines, dashboards and debugging workflows that make AI behavior inspectable.
• Own backend systems from architecture and design docs through production rollout, reliability, monitoring and iteration.
• Partner with product, ML and infrastructure engineers to make evals a first-class part of how Glean ships AI features.
• Connect eval results, customer feedback, regression analysis and engineering workflows into concrete product improvements.
• Build systems that balance speed, reliability, enterprise security and cost across modern cloud-native environments.
• Mentor other engineers, raise the bar for technical design, and help shape the engineering culture of Glean's Bangalore AI quality team.

What you'll need

• 6+ years of software engineering experience building backend systems, infrastructure, distributed systems or data platforms.
• Strong coding skills in Go, Python, Java, C++ or similar languages, with an emphasis on reliability, scale and well-tested components.
• Comfortable working with distributed data pipelines, production services, observability systems or cloud-native infrastructure.
• Analytically rigorous, caring about whether metrics reflect real user experience rather than just whether dashboards look good.
• Enjoy customer-focused, cross-functional environments and willing to take on whatever is most impactful for the company.
• Care deeply about quality, both in the systems you build and in the AI product you help measure and improve.

Nice to have

• Experience with LLM applications, evals, tracing, data warehouses, workflow orchestration or ML infrastructure.

Details

• Location: in person in Bangalore, India.
• An AI-focused exercise or discussion is part of the interview process.