Staff Engineer, Distributed Storage and HPC & AI Infrastructure

Overview

Staff Engineer to operate, scale and optimize multi-petabyte storage systems for large AI training and inference workloads, covering parallel filesystems, object stores and Kubernetes-native storage platforms.

What you'll do

• Architect and implement the technical strategy and storage roadmap for Together AI, driving high-performance architectural decisions as the GPU fleet scales.
• Engineer and scale multi-petabyte AI/ML storage systems by integrating Vast, Weka, and Ceph, with cost optimization through automated tiering and lifecycle policies.
• Develop intelligent caching and tiered storage architectures to achieve extreme IOPS and cluster-wide throughput at GPU scale for training and inference workloads.
• Tune storage isolation at the L2/L3 network layers to ensure secure, production-grade multi-tenancy for storage clients.
• Code Kubernetes storage operators and controllers to enable automated provisioning, self-service abstractions, and quota enforcement.
• Engineer end-to-end data paths to achieve 10+ GB/s per GPU node, architect multi-tier caching for model weights and datasets, tune parallel filesystems using advanced profiling, and scale storage infrastructure across thousands of nodes.
• Optimize end-to-end data paths through advanced benchmarking and profiling, contributing high-impact code to open-source storage projects and internal tooling.

What you'll need

• 8+ years in storage engineering, managing distributed storage at multi-petabyte scale.
• Proven track record deploying and operating high-performance storage for GPU/HPC clusters.
• Deep Kubernetes and cloud-native storage experience in production environments.
• Strong coding skills in Go and Python with demonstrated ability to build production-grade systems and tooling.
• BS/MS in Computer Science, Engineering, or equivalent practical experience.
• History of technical leadership: designing systems that significantly improved performance, reliability (99.999%+ uptime), or cost efficiency.
• Distributed Storage Systems: deep expertise in either of Ceph, WekaFS, Lustre, Vast, GPFS, or similar parallel filesystems at multi-petabyte scale.
• Object Storage: production experience with S3, MinIO, Ceph, or R2, including performance optimization and cost management.
• Kubernetes Storage: CSI drivers, StatefulSets, PersistentVolumes, storage operators, and custom controllers.
• Storage optimization for GPU workloads, RDMA/InfiniBand networking, and parallel filesystem optimization (TB/s aggregate cluster throughput, line saturation).
• Programming: Go and Python for automation, operators, and tooling.
• Infrastructure as Code: Terraform, Ansible, Helm, GitOps (ArgoCD).
• Linux Storage Stack: advanced knowledge of filesystems (ext4, xfs), LVM, NVMe optimization, RAID configurations.
• Observability: Prometheus, Grafana, Thanos architecture and operations.

Nice to have

• GPU Direct Storage (GDS), NVMe-oF, storage networking, RDMA implementations.
• ML/AI storage patterns (model weights, checkpointing, dataset caching).
• Storage benchmarking and profiling tools (fio, iperf3, iostat, blktrace).

Details

• Location: Bangalore, India.