Sr. Principal Software Engineer

Overview

Lead the optimization and deployment of high-performance large language model inference across data center, edge, and embedded platforms. The role focuses on improving model efficiency, latency, throughput, and production reliability.

What you'll do

• Optimize and deploy LLM inference pipelines.
• Own inference runtimes across data center, edge, and embedded platforms.
• Extend and tune vLLM, TensorRT-LLM, llama.cpp, and QAIRT.
• Develop custom CUDA kernels and profile inference performance.
• Implement quantization strategies including INT8, INT4, FP4, FP8, AWQ, and GPTQ.
• Optimize KV cache performance, batching, speculative decoding, latency, and throughput.
• Adapt inference runtimes for constrained and embedded environments.

What you'll need

• Experience optimizing ML inference performance in production.
• Deep understanding of GPU architecture and memory hierarchies.
• Hands-on CUDA and low-level performance-tuning experience.
• Experience deploying models beyond research environments.
• Knowledge of vLLM, TensorRT-LLM, llama.cpp, or QAIRT.
• Knowledge of KV cache optimization and memory layout design.

Details

• Location: United States.
• Remote work is available.
• Salary range: USD 185,000–280,000 yearly.