Senior Data Engineer-1

Overview

Mastercard is seeking a Senior Data Engineer to join its team in Hyderabad, India. This experienced individual-contributor role owns end-to-end pipeline delivery, mentors peers, and delivers complex data solutions across functional teams.

What you'll do

• Own the end-to-end design, development, and delivery of complex batch and real-time data pipelines using Databricks, Spark, Python, and PySpark.
• Drive dbt-based transformation architecture with rigorous Test-Driven Development (TDD) and modular design.
• Design and own CI/CD pipelines using GitLab and Jenkins, and champion DataOps practices across the team.
• Administer and optimize Databricks objects and the platform using Databricks Asset Bundles, managing performance, access, and cost.
• Establish data observability frameworks covering quality, freshness, lineage, and anomaly detection.
• Lead the design of real-time data streaming pipelines, such as using Kafka and Spark Structured Streaming.
• Mentor junior and mid-level engineers, and lead code and design reviews.
• Partner with stakeholders on requirements, delivery planning, and technical trade-offs.
• Plan and execute deployments, migrations, and upgrades with minimal disruption to operations.
• Abide by Mastercard’s security policies and practices.
• Ensure the confidentiality and integrity of the information being accessed.
• Report any suspected information security violation or breach.
• Complete all periodic mandatory security trainings in accordance with Mastercard’s guidelines.

What you'll need

• Bachelor's degree in computer science or a related technical field.
• 5+ years of data engineering experience.
• Deep hands-on experience with Databricks, including administration, Spark, and Python.
• Strong background in PySpark and distributed data processing.
• Proven track record using dbt for robust, testable transformation workflows following TDD.
• Familiarity with Databricks Asset Bundles for object deployment and version control.
• Strong CI/CD expertise with GitLab and Jenkins, and DataOps expertise.
• Experience with real-time data processing and streaming pipelines.
• Expert SQL development and strong cloud infrastructure experience with AWS, Azure, or GCP.
• Demonstrated experience mentoring engineers and leading technical delivery.

Nice to have

• Master's degree.
• Experience in banking, e-commerce, credit cards or payment processing.
• Exposure to both SaaS and premises-based architectures.
• Post-secondary degree in computer science, mathematics, or quantitative science.

Details

• Location: Hyderabad, India.
• The role is an experienced individual-contributor position.