Lead / Manager - Data Engineering

Overview

Lead/Manager for Data Engineering to build and manage on-premise data engineering solutions, including scalable pipelines and team delivery.

What you'll do

• Lead and manage a team of Data Engineers and provide technical guidance and mentorship.
• Design, develop, and optimize scalable data pipelines in an on-premise environment.
• Build ETL/ELT workflows using Python, Spark, SQL, and Airflow.
• Design and manage complex Apache Airflow DAGs for orchestration.
• Troubleshoot pipeline failures, performance issues, and data quality challenges.
• Conduct code reviews and ensure adherence to engineering best practices.
• Drive technical design, planning, monitoring, and end-to-end delivery.

What you'll need

• 8+ years of experience in data engineering, building and managing solutions.
• Strong hands-on expertise in Python.
• Strong experience with Apache Spark/PySpark.
• Advanced SQL skills including complex joins, CTEs, window functions, subqueries, and optimization.
• Strong experience with Apache Airflow for workflow orchestration and scheduling.
• Experience working with on-premise data environments and ETL/ELT pipeline architecture.
• Proven experience leading/managing data engineering teams.

Nice to have

• Experience with large-scale enterprise data platforms.
• Experience with data warehouse concepts and data modelling.
• Experience with on-premise Hadoop/data ecosystems.
• Experience migrating or modernizing legacy/on-premise data platforms.

Details

• Location: Hyderabad, TS, India.
• Environment: on-premise.