Staff Software Engineer, Inference

Overview

Build and maintain distributed inference systems that serve Claude to millions of users worldwide.

What you'll do

• Design, build, and maintain distributed systems for large-scale Claude inference.
• Create resilient systems that adapt in real time to real-world events.
• Develop intelligent request routing, load balancing, and traffic management for accelerators.
• Improve compute efficiency using autoscaling and orchestration across different workloads.
• Build and operate production-grade deployment pipelines for releasing new models.
• Provide high-performance inference infrastructure for next-generation model development.
• Integrate new AI accelerator platforms and support new model architectures.

What you'll need

• Proficiency in Python or Rust.
• Production experience building and operating distributed systems.
• Working knowledge of containerized infrastructure (e.g., Kubernetes).
• Working knowledge of at least one major cloud platform (AWS, GCP, or Azure).
• Results-oriented mindset with flexibility and impact focus.
• Willingness to take on work beyond the job description.
• Interest in learning about machine learning systems and infrastructure.
• Preference for environments where technical excellence drives business and research outcomes.

Details

• Hybrid work: expected to be in an office at least 25% of the time.