Overview
Senior technical lead to architect, deliver and manage enterprise-scale AWS and Databricks data platforms and modernization initiatives, combining hands-on engineering with team leadership.
What you'll do
- Lead the architecture and implementation of enterprise-scale data platforms across AWS and Databricks, using Medallion, Lakehouse and Data Warehouse patterns.
- Define target-state architecture, data modeling standards and reusable engineering frameworks, ensuring scalability, performance, security and cost efficiency.
- Drive architectural decisions and design data platforms supporting advanced analytics, AI/ML and Generative AI workloads.
- Design scalable data ingestion and processing frameworks for batch, streaming, real-time and CDC pipelines using AWS services and Databricks.
- Establish engineering best practices for data integration, quality, performance optimization, observability and production support.
- Drive platform optimization to improve processing efficiency, reliability, scalability and cloud cost efficiency.
- Lead, mentor and manage data engineering teams, providing technical guidance, career development and continuous learning opportunities.
- Own team execution, including sprint planning, task allocation, effort estimation, progress tracking and delivery commitments.
- Drive engineering excellence through architecture reviews, code reviews, technical mentoring and resolution of complex engineering challenges.
- Own end-to-end technical delivery, from requirements gathering and solution design to development, deployment and production stabilization.
- Establish CI/CD pipelines, automated testing, reusable frameworks and engineering standards across data platforms.
- Proactively manage technical risks, resolve delivery blockers and ensure adherence to timelines and quality standards.
- Define and implement enterprise data governance frameworks covering cataloging, lineage, stewardship, data quality, metadata management and compliance.
- Establish secure data access patterns and governance standards leveraging tools such as Atlan, Unity Catalog and AWS Lake Formation.
- Engage with clients as a trusted technical advisor, driving data strategy, architecture roadmaps and solution design discussions.
- Translate business requirements into technical solutions and communicate architecture recommendations to senior stakeholders.
- Collaborate with cross-functional teams, manage stakeholder expectations and identify opportunities for platform optimization and expansion of data and AI capabilities.
What you'll need
- 14+ years of experience in Data Engineering, Data Architecture and Cloud Data Platforms.
- Proven leadership in large-scale enterprise data modernization initiatives.
- Strong hands-on expertise in AWS data services, including S3, Glue, EMR, Lambda, Redshift, Lake Formation, Athena and DMS.
- Advanced Databricks experience across Delta Lake, Unity Catalog, Databricks SQL and Delta Live Tables.
- Deep expertise in data architecture and engineering, including Data Warehousing, Data Modeling (Dimensional, Data Vault), MDM, Data Quality, Metadata Management and Data Governance.
- Strong understanding of batch, streaming, real-time and CDC architectures, with experience in technologies such as Kafka, Kinesis, Apache Airflow and MWAA.
- Proven experience leading and managing data engineering teams, with strong capabilities in technical mentoring, people development, architecture decision-making and end-to-end delivery ownership.
- Experience implementing CI/CD pipelines, infrastructure automation and engineering best practices for scalable, secure and high-performance data platforms.
- Hands-on experience enabling AI/ML and Generative AI workloads, including AI-ready data foundations, feature layers, vector storage and secure data access patterns.
- Strong client-facing and stakeholder management skills, with the ability to lead technical discussions, communicate architecture recommendations and drive enterprise data strategy.
Nice to have
- AWS, Databricks and/or Snowflake certifications.
Details
- Locations: Mumbai, Bangalore, Hyderabad or Gurgaon.
- Hybrid work model: 3 days a week in office.
Read the full description and apply on the company’s own careers page.