Lead Data Engineer

Overview

Lead Data Engineer for Mastercard’s AI&DPE Data Engineering platform team, building and evolving large-scale backend data systems, batch/real-time pipelines, and AI-enabled services.

What you'll do

• Design, develop, and maintain backend services and data pipelines using Scala, Python, and Java.
• Build and optimize batch and streaming workloads using Cloudera Data Platform (CDP) and Spark.
• Use Cloudera Manager for configuration, monitoring, performance tuning, and operational stability.
• Design and implement datamarts and curated datasets with strong data integrity, performance, and reliability.
• Design and integrate AI-powered agents for anomaly detection and operational intelligence workflow automation.
• Build and support data/compute workloads in AWS and leverage Databricks for large-scale processing.
• Provide reliable backend data interfaces for downstream consumers and document technical designs.

What you'll need

• BS/BA degree in Computer Science, Engineering, Information Systems, or a related field.
• Strong hands-on experience with Scala and Python/Java in backend or data-intensive systems.
• Experience working with Cloudera Data Platform (CDP) and Spark.
• Familiarity with Cloudera Manager for cluster administration, monitoring, or troubleshooting.
• Strong understanding of data modeling, distributed systems, and large-scale data processing.
• Excellent problem solving skills and ability to work independently in complex environments.

Details

• Location: Pune, India.