Senior System Software Engineer - Local AI Automation

Overview

Senior System Software Engineer on NVIDIA's Local AI Automation team, building and maintaining system-level software and infrastructure to deploy, benchmark and qualify AI applications and models.

What you'll do

• Design, implement and maintain system software and infrastructure to run AI workloads through inference backends such as Llama.cpp, Ollama, PyTorch, vLLM, WinML, TRT-RTX and others.
• Scope system requirements for deploying AI applications, benchmarks and models in automation, and develop solutions to measure correctness, functionality, reliability and overall system performance.
• Develop infrastructure to download AI models from various sources, manage a local model repository, and build automated synchronization mechanisms to keep it up to date.
• Analyze large datasets and system workload metrics, implement data processing infrastructure to turn raw data into actionable information for developers, and design useful visualizations.
• Collaborate with Local AI developers to implement system-level instrumentation, diagnostics and debugging capabilities that help quickly isolate issues.
• Design, implement and maintain infrastructure for automated build, integration, deployment and qualification across inference frameworks and environments.
• Understand existing infrastructure and system software architecture, debug issues in current frameworks, identify performance bottlenecks, and devise solutions to improve system efficiency and scalability.

What you'll need

• 5+ years of experience with a B.Tech or higher degree in Computer Science, Information Technology, Software Engineering or a related field.
• Strong analytical, systems debugging and problem-solving abilities, with the capacity to multitask effectively in a dynamic environment.
• 5+ years of software development experience with strong programming skills in C/C++, C#, Java or another systems programming language.
• Exposure to at least one scripting language such as Python.
• Experience with system APIs, multithreaded software, process and resource management, debugging or performance analysis.
• Familiarity with databases and SQL, source control systems, and CI/CD infrastructure such as Git, Perforce and Jenkins.
• Outstanding written and oral communication skills, enabling effective collaboration with management and engineering teams.

Nice to have

• Experience building robust system software, runtime infrastructure or scalable backend components.
• Experience with system profiling, performance optimization, debugging and telemetry/observability infrastructure, including analysis of Nsight Systems traces to understand system state and kernel performance.
• Hands-on background with Linux/Windows system programming, Git CI/CD, databases/SQL, Kubernetes, Docker, and observability tools such as Grafana or Kibana.
• Hands-on experience integrating or debugging inference frameworks such as Llama.cpp, Ollama, PyTorch, vLLM, WinML or TRT-RTX.
• Experience understanding CPU/GPU resource utilization, memory management, concurrency and performance characteristics of AI workloads.

Details

• Location: Pune, India.