Performance Engineer, Inference Systems

Overview

Performance Engineer for Inference Systems, working on Anthropic’s inference fleet that powers Claude across products and cloud platforms. You’ll investigate and improve system performance while treating correctness as part of performance.

What you'll do

• Run cross-layer performance investigations across throughput, latency, and reliability.
• Size gaps between actual fleet performance and theoretical performance rooflines.
• Own and improve the correctness evaluation pipeline across hardware platforms, numerics, and serving configurations.
• Build observability, dashboards, and modeling tools for throughput, latency, cost, reliability, correctness, and their interactions.
• Partner with kernel, serving, routing, autoscaling, and capacity teams to prioritize and land optimizations.
• Stack-rank a surface area of opportunities by impact and effort and de-prioritize lower-value work.

What you'll need

• Hands-on performance engineering experience including profiling, roofline analysis, and latency/throughput optimization in production systems.
• Proficiency in Python, including the ability to read, instrument, and contribute to large production codebases.
• Solid data analysis skills such as SQL and pandas to turn telemetry into clear findings.
• Ability to communicate quantitative results clearly in writing to influence teams.
• Interest in correctness as an engineering discipline, including numerics, evaluation design, and regression detection.