DA Lead Data Engineer

Overview

Lead Data Engineer responsible for designing, implementing, and evaluating scalable data ingestion and processing solutions.

What you'll do

• Design performant data ingestion pipelines from multiple sources.
• Develop scalable, reusable data ingestion frameworks.
• Integrate source systems with target data repositories.
• Maintain data quality and consistency across pipelines.
• Process data using event-based and streaming technologies.
• Analyze requirements and propose solutions.
• Evaluate tools against customer requirements and present proofs of concept.

What you'll need

• At least 5 years of hands-on experience with PySpark, Python, Spark, and Scala.
• At least 5 years of Snowflake or Databricks cloud data warehouse implementation experience.
• At least 2 years of ETL experience across multiple sources and data formats.
• At least 2 years of experience with Python or Spark big data modeling techniques.
• At least 2 years of CI/CD experience using Jenkins, Terraform, Docker, AWS, and Kubernetes.
• Bachelor’s degree in computer science, engineering, or a related field.
• Experience with data validation, quality, integrity, and data warehousing concepts.

Nice to have

• Pharma industry experience.

Details

• Location: Bengaluru, India.
• Work schedule: 2:00 PM to 11:00 PM IST.