Sr. Manager; Data Engineering

Overview

Lead a team of platform engineers responsible for building, operating and modernizing critical platform services that power Visa's enterprise data, analytics and AI ecosystem within the Data Trust & Platform Engineering (DTPE) organization.

What you'll do

• Lead the design, development, deployment and operation of large-scale platform engineering solutions supporting analytics, AI, data processing and enterprise workloads.
• Drive architecture and implementation of highly scalable, reliable, secure and cost-efficient platform services.
• Partner closely with architects, principal engineers and product stakeholders to define technical roadmaps and engineering priorities.
• Lead platform modernization initiatives across hybrid-cloud environments, cloud-native architectures, container platforms and developer self-service capabilities.
• Establish engineering standards and best practices covering scalability, resiliency, observability, automation, security and performance.
• Lead adoption of cloud-native technologies across AWS, Azure, Kubernetes, Databricks and modern platform engineering ecosystems.
• Drive development of reusable platform services for data engineering, analytics, AI/ML workloads and enterprise applications.
• Enable large-scale distributed compute and storage platforms supporting petabyte-scale data processing.
• Build and operationalize data platform capabilities including orchestration, streaming, metadata management, governance and workload automation.
• Improve platform efficiency, utilization and cost optimization through automation and infrastructure engineering.
• Own engineering execution, service delivery, platform availability and customer commitments.
• Drive continuous improvements in reliability, scalability, incident reduction and operational maturity.
• Lead production readiness reviews, incident management, root-cause analysis and remediation efforts.
• Define and track operational KPIs including service availability, latency, reliability, customer satisfaction and engineering productivity.
• Champion Site Reliability Engineering (SRE) principles, observability, alerting and capacity planning practices.
• Drive adoption of GenAI-powered engineering capabilities including AI-assisted development, code generation, intelligent troubleshooting and operational automation.
• Partner with AI and platform teams to enable secure and governed GenAI infrastructure and services.
• Identify opportunities to leverage LLMs, retrieval-augmented generation (RAG), AI agents and automation frameworks to improve engineering productivity and customer experience.
• Promote responsible AI practices, governance controls, security and compliance requirements for enterprise AI solutions.
• Lead, mentor and develop high-performing engineering teams across multiple locations.
• Drive hiring, talent development, performance management and career growth for engineers and technical leads.
• Foster a culture focused on technical excellence, innovation, accountability, inclusion and continuous learning.
• Encourage knowledge sharing, technical mentorship and engineering craftsmanship across teams.
• Partner with Data Engineering, Product, Infrastructure, Security, Operations and Architecture teams to deliver scalable platform capabilities.
• Act as an advocate for platform customers by improving platform usability, self-service adoption and developer experience.
• Collaborate with business and engineering stakeholders to prioritize investments and deliver measurable business outcomes.
• Communicate engineering progress, risks, operational health and roadmap updates to leadership.

What you'll need

• 10+ years of software engineering, platform engineering, cloud engineering, infrastructure engineering or data platform engineering experience.
• 3+ years of experience leading engineering teams and delivering large-scale enterprise platforms.
• Strong hands-on experience designing and operating distributed systems in production environments.
• Experience with cloud platforms such as AWS and/or Azure.
• Experience with container orchestration technologies such as Kubernetes and cloud-native architectures.
• Strong knowledge of software engineering fundamentals, system design, networking, storage, security and performance optimization.
• Experience building highly available, fault-tolerant and scalable distributed services.
• Experience driving automation using Infrastructure as Code (Terraform, CloudFormation, etc.) and CI/CD practices.
• Demonstrated experience leveraging GenAI, machine learning or intelligent automation technologies to improve engineering outcomes.

Nice to have

• Strong knowledge of AWS services such as EKS, EC2, S3, EMR, Glue, Lambda, ECS and IAM.
• Strong knowledge of Databricks services.
• Experience with Azure services including AKS, Data Factory, Synapse, Fabric, Storage and Entra ID.
• Experience building hybrid-cloud platform solutions spanning on-premises and cloud environments.
• Expertise in Kubernetes, container management, service mesh and cloud-native infrastructure.
• Experience operating enterprise-scale data platforms.
• Knowledge of Hadoop, Spark, Trino/Presto, Kafka, Databricks, Snowflake, Delta Lake, Iceberg or similar technologies.
• Experience supporting large-scale batch, streaming, AI and analytics workloads.
• Understanding of metadata, governance, data quality, lineage, privacy and access management capabilities.
• Experience with developer platforms and self-service engineering models.
• Strong understanding of observability platforms such as Prometheus, Grafana, Datadog, Splunk, ELK or OpenTelemetry.
• Experience implementing SRE practices including SLOs, SLIs, error budgets, reliability engineering and operational automation.
• Strong troubleshooting and performance tuning experience across distributed environments.
• Understanding of LLM architectures and enterprise GenAI platforms.
• Experience with Retrieval-Augmented Generation (RAG), vector databases, embeddings, AI agents and prompt engineering.
• Familiarity with GenAI frameworks such as LangChain, LangGraph, Semantic Kernel, Crew AI, Autogeny or equivalent.
• Understanding of responsible AI, model governance, observability, security and privacy requirements.

Details

• Location: Bengaluru, India.
• Work mode: at least 3 days in office; specific expectations to be confirmed by the Hiring Manager.
• All roles require digital fluency, including the ability to work with emerging technologies such as Generative AI tools (e.g. ChatGPT, Microsoft Copilot).
• Leadership expectations: technical leadership and engineering excellence; customer-centric platform mindset; execution with excellence; data-driven decision making; platform reliability and operational excellence; innovation through automation and AI; talent development and coaching; collaboration across teams; ownership and accountability; continuous learning and growth.