Staff+ Software Engineer, Inference Runtime

Overview

Staff+ Software Engineer (Inference Runtime) to lead the shared, accelerator-agnostic core of the inference serving stack that powers Claude.

What you'll do

• Set technical direction for the runtime’s architecture and roadmap.
• Own and evolve the accelerator-agnostic runtime interfaces, boundaries, and build structure.
• Work hands-on in a performance-sensitive Rust and Python codebase.
• Drive efficient accelerator utilization, scheduling, and memory management across GPU, TPU, and Trainium.
• Build validation mechanisms using partitioned builds, change-scoped testing, and canary/shadow/rollback.
• Partner on build systems, toolchains, and compilers to meet inference performance and correctness requirements.
• Mentor engineers via design and code review, and raise the technical bar.

What you'll need

• Deep background in systems engineering or ML infrastructure with hands-on performance and systems debugging.
• Real depth in at least one accelerator ecosystem (CUDA/GPU, TPU, or Trainium/AWS Neuron).
• Significant software engineering experience in high-performance, large-scale distributed systems serving millions of users.
• Experience defining engineering metrics and improving measurable outcomes (e.g., SLOs, escape rates, release times, latency/throughput).
• Experience driving technical alignment across organizational boundaries.
• Strong written and verbal communication to influence technical direction without formal authority.

Details

• Annual compensation range: $405,000–$485,000 USD.
• Work mode: hybrid; staff expected in one of the offices at least 25% of the time.
• Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience.
• Visa sponsorship: offered, with reasonable efforts made if an offer is made.