Senior Machine Learning Engineer, ML Infrastructure- Online

Overview

Design and evolve Unity Vector's online model inference platform, building reliable infrastructure for serving, optimizing, and monitoring machine learning models in production.

What you'll do

• Design and operate large-scale online inference infrastructure that serves production ML models with low latency and high reliability, such as PyTorch, Triton Inference Server, Kubernetes, GKE, Ray, or similar distributed serving frameworks.
• Develop infrastructure that supports distributed training workflows using technologies such as PyTorch, Ray Data, and Ray Train, etc.
• Integrate ML pipelines with workflow orchestration systems (e.g., Flyte, Airflow, or similar) to enable reliable multi-stage training workflows.
• Optimize model performance through model compilation, GPU/CPU utilization improvements, request scheduling, kernel fusion, and runtime-level tuning.
• Improve observability of ML systems through latency, throughput, error-rate, cost, saturation, and model-health monitoring.
• Partner closely with ML engineers to support faster model iteration while maintaining production safety, scalability, and cost efficiency.
• Improve the reliability and reproducibility of model serving workflows, including model packaging, artifact validation, compatibility testing, and deployment automation.
• Lead architectural improvements that make the online ML platform more robust, user-friendly, scalable, and cost-efficient.

What you'll need

• Experience building and operating production-grade online ML inference systems, such as NVIDIA Triton Inference Server, TorchServe, Ray Serve, TensorFlow Serving, or similar systems.
• Experience optimizing inference workloads using techniques such as dynamic batching, model compilation, quantization, GPU acceleration, GPU kernel optimization, caching, or runtime tuning.
• Strong experience with distributed systems, Kubernetes, autoscaling, service reliability, and production observability.
• Strong programming skills in Python, with practical experience working on production ML systems and high-scale services.
• Experience with PyTorch and modern model deployment workflows, including model packaging, validation, and serving lifecycle management.
• Experience designing infrastructure for safe model rollout, canary testing, A/B experimentation, and automated rollback.
• Strong systems thinking, with the ability to reason about latency, throughput, reliability, scalability, and cost tradeoffs in online systems.
• Proven ability to lead technical direction and influence architectural decisions across teams without formal authority.
• Fluent professional English for verbal and written communication with colleagues and partners worldwide.

Details

• Location: Remote, Washington, USA.
• Base salary ranges by zone: Zone A $210,300 - $273,400; Zone B $187,200 - $243,300; Zone C $165,600 - $215,200. Final offer depends on geographic location, experience, professional background, and skill set.
• Eligible for equity awards and company incentive plans (such as annual discretionary bonuses or sales commissions).