Software Engineer 4/5 – Model Serving Systems, AI Platform

Overview

Software Engineer (4/5) for Netflix’s Model Serving Systems team, building computation and infrastructure to serve AI/ML and LLM model inference at scale.

What you'll do

• Develop and expand compute infrastructure to support growing AI needs.
• Provide building blocks for real-time AI/ML model inference and serving at scale.
• Build foundational abstractions to ensure consistency between online and offline systems.
• Build model serving infrastructure for LLMs and other large foundation models.
• Partner cross-functionally with engineers, product managers, ML engineers, and data/research scientists.
• Enable application of ML in new business areas and drive AI/ML innovation.

What you'll need

• Experience building high-traffic distributed services and infrastructure for online ML model inference.
• Experience supporting large-scale ML models with high availability and performance.
• Skills for scalable model-serving solutions for generative models and LLMs, including reducing latency and costs.
• Proficiency in object-oriented programming, preferably Java.
• Experience deploying ML models using tools like Triton Inference Server, TensorRT, and Docker.
• Experience working with public cloud platforms such as AWS, Azure, or GCP.
• A BS/MS in Computer Science, Applied Math, Engineering, or a related field.

Details

• Location: USA - Remote.
• Compensation range: $466,000.00 - $750,000.00 annually.