Senior, Data Engineer

Overview

Senior Data Engineer role on Walmart's newly formed Global Data Foundation (GDF) platform team, building reliable, high-throughput data pipelines and distributed processing systems at Walmart scale.

What you'll do

• Design, build, and operate batch and streaming data pipelines using Spark, Java, and Python, processing data at Walmart scale.
• Apply solid software engineering fundamentals (clean architecture, automated testing, CI/CD, code review discipline) to data engineering work.
• Apply system design and low-level design (LLD) rigor to data and platform problems, evaluating dependencies, trade-offs, failure modes, scalability, and cost before writing code, with support from Staff/Principal engineers on the most ambiguous or cross-team problems.
• Flex into platform/software engineering work when needed, contributing to services, APIs, and distributed systems that data pipelines depend on.
• Apply AI, GenAI, and agentic capabilities to data platform problems: embeddings/vector search for data discovery, LLM-assisted data quality and anomaly detection, agentic pipeline monitoring, AI-assisted schema and lineage documentation.
• Partner with data scientists and ML engineers to build feature pipelines, model-serving data paths, and the data infrastructure that MLOps depends on.
• Own data quality, observability, and governance for the pipelines and datasets you build, with a strong Operational Excellence / Engineering Excellence discipline.
• Participate actively in architecture and design reviews spanning data engineering and platform engineering domains.
• As a Senior engineer, bring technical judgment to design and code reviews within your immediate team and area of ownership, mentor early-career engineers, and help raise the bar for engineering quality without needing a formal management title.
• Stay current on emerging Big Data, cloud, and AI/agentic technologies, and propose where they could pragmatically improve reliability, productivity, or business value within your team's scope.

What you'll need

• 5+ years of professional software engineering experience with hands-on data engineering at scale.
• Strong proficiency in Java and Python.
• Strong working knowledge of Spark for distributed data processing.
• Operational Excellence (OE) and Engineering Excellence (EE) mindset: SLO/SLA ownership, incident management and blameless post-incident reviews, proactive reliability and quality metrics, and continuous improvement built into how you ship.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
• Solid experience with the Big Data ecosystem (Spark, Hadoop/HDFS, Hive, Kafka, workflow orchestration such as Airflow or equivalent), including batch and streaming processing and large-scale data transformation.
• Solid data platform fundamentals: data modeling, ETL/ELT pipeline design, data quality and observability, data governance, and integration with lakehouse architectures (e.g., Delta Lake, BigQuery, Databricks).
• Full exposure to AI and Agentic technologies: hands-on experience integrating model APIs and GenAI/LLM tooling, building or operating agentic workflows (tool-calling agents, multi-step autonomous systems), working with embeddings/vector search, and applying MLOps practices (feature stores, model lifecycle, retraining pipelines).
• Working literacy in platform engineering: microservices, API design, Kubernetes or equivalent orchestration, CI/CD, and cloud-native practices.
• Solid distributed systems fundamentals: event-driven patterns, service contracts, asynchronous processing, and fault tolerance at scale, paired with solid system design and low-level design (LLD) skills.
• Strong software engineering craftsmanship: testing strategy, code review standards, design patterns, and system design for high-throughput data systems.
• Proven ability to translate reasonably well-scoped data/business problems into clear technical execution plans, particularly within a newly formed team, escalating cross-team ambiguity to Staff/Principal engineers as needed.
• Strong ownership mindset and technical judgment; comfortable operating with autonomy within your team in a 0-to-1 environment.
• Minimum qualification, Option 1: Bachelor's degree in Computer Science and 3 years' experience in software engineering or related field.
• Minimum qualification, Option 2: 5 years' experience in software engineering or related field.
• Minimum qualification, Option 3: Master's degree in Computer Science and 1 year's experience in software engineering or related field, plus 2 years' experience in data engineering, database engineering, business intelligence, or business analytics.

Nice to have

• Additional experience with Go, C++, or Scala.

Details

• Location: Block-1, Prestige Tech Pacific Park, Sy No. 38/1, Outer Ring Rd, Kadubeesanahalli, India (Bangalore).
• Work mode: office-based for daily work at the Bangalore/Chennai office, with flexibility for associates to manage personal lives.