Senior Software Engineer, AI Infrastructure Solutions

Overview

Senior Software Engineer on NVIDIA's AI Infrastructure Solutions team, designing and operating AI infrastructure that powers LLM-based applications and autonomous agents across the enterprise.

What you'll do

• Design, build, and operate AI infrastructure solutions that power LLM-based applications and autonomous agents across the enterprise.
• Develop agentic AI workflows and multi-agent systems, including tool calling, planning, memory, retrieval-augmented generation (RAG), and orchestration using frameworks such as LangChain, LangGraph, or similar.
• Integrate and serve LLMs through APIs and inference endpoints, optimizing for latency, throughput, cost, and reliability.
• Containerize applications with Docker and deploy and scale them on Kubernetes, including Helm charts, autoscaling, and GPU-enabled workloads.
• Take applications to L3 level: own production readiness, advanced troubleshooting, root-cause analysis, performance tuning, and escalation support for complex issues beyond L1/L2.
• Define and maintain observability for AI systems, including logging, metrics, tracing, and monitoring of agent behavior, LLM quality, and system health.
• Build and maintain CI/CD pipelines for automated build, test, and deployment of AI applications and infrastructure.
• Develop high-performance backend services and RESTful APIs (Python and/or .NET Core) that expose AI capabilities to internal applications.

What you'll need

• Bachelor's or equivalent experience in Computer Science, Engineering, or a related field.
• 5+ years of software engineering experience, with 2+ years working on AI/LLM-based solutions.
• Hands-on experience building agentic AI and autonomous agent systems, including tool use, multi-agent orchestration, and RAG pipelines.
• Strong experience with LLMs and their integration (OpenAI-compatible APIs, NVIDIA NIM, Hugging Face, or similar), prompt engineering, and evaluation.
• Strong proficiency in Python.
• Solid experience with Docker and Kubernetes, including deploying, scaling, and troubleshooting containerized workloads in production.
• Proven experience supporting applications at L3 level, including incident management, debugging, performance optimization, and root-cause analysis.
• Experience with CI/CD tools (GitHub Actions, Azure DevOps, GitLab CI, Jenkins) and Git-based workflows.

Nice to have

• Experience with C#/.NET Core and ASP.NET Core for REST API development.
• Experience with GPU-accelerated workloads, NVIDIA AI Enterprise, NVIDIA NIM, Triton Inference Server, or NeMo.
• Familiarity with MCP (Model Context Protocol) and agent-to-agent communication patterns.
• Experience with LLM evaluation, guardrails, and safety frameworks.
• Contributions to open-source AI or infrastructure projects.

Details

• Location: Pune, India.