Lead Data Engineer (10+ years with MS Fabric, Azure, ADF, Data Platforms only)

Overview

Lead a growing team of data engineers as a hands-on technical leader, setting engineering standards and building enterprise-scale, AI-first data platforms and data products.

What you'll do

• Provide hands-on technical leadership for a growing team of data engineers.
• Set engineering standards and reusable patterns across architecture, coding, testing, data modeling, documentation, observability, security, and deployment.
• Lead architecture discussions, technical design reviews, and code reviews.
• Mentor engineers and develop technical depth, ownership, problem-solving, and engineering judgment across the team.
• Translate complex initiatives into clear technical approaches and guide teams from design through production.
• Lead the architecture and engineering of scalable, secure, and governed enterprise data platforms.
• Design solutions across lakehouse, data warehouse, data lake, streaming, and data-product architectures.
• Build batch, incremental, near-real-time, and event-driven data processing solutions.
• Create reusable engineering frameworks for ingestion, transformation, orchestration, data quality, metadata, and data delivery.
• Design scalable data models supporting operational, analytical, reporting, and AI use cases.
• Evaluate technology choices based on performance, scalability, reliability, security, cost, maintainability, and business value.
• Champion the use of AI across the engineering lifecycle, including requirements, architecture, development, testing, documentation, troubleshooting, and operations.
• Promote spec-driven development using well-defined requirements, data contracts, acceptance criteria, and automated tests.
• Design data platforms and products ready to support AI agents, RAG, semantic models, machine learning, and conversational analytics.
• Establish automated data quality, reconciliation, validation, and testing practices.
• Build observability across data pipelines, datasets, APIs, and platform services.
• Lead root-cause analysis for significant issues and ensure corrective actions address systemic problems.
• Design secure, scalable integration patterns across enterprise applications, data platforms, and external data sources.
• Develop and guide the implementation of REST and GraphQL APIs.
• Design event-driven and serverless integration solutions.
• Establish standards for API lifecycle management, authentication, versioning, monitoring, and governance.
• Build data governance, lineage, metadata, data quality, security, privacy, and auditability into platform designs.
• Apply appropriate engineering rigor to KYC, UCC, public-record, and other regulated or sensitive data domains.
• Partner with product, architecture, analytics, AI, operations, and business teams to translate opportunities into scalable technical solutions.

What you'll need

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline, or equivalent practical experience.
• 10+ years of experience in data engineering, software engineering, data platforms, or related disciplines.
• Demonstrated experience providing technical leadership, mentoring engineers, and leading complex initiatives from architecture through production.
• Strong understanding of modern data platform and distributed data architecture principles.
• Advanced SQL skills and strong expertise in data modeling, database design, and performance optimization.
• Deep hands-on experience designing and building scalable ETL/ELT pipelines and reusable data engineering frameworks.
• Significant experience with one or more modern cloud data platforms, such as Databricks, Microsoft Fabric, Snowflake, or comparable platforms.
• Strong programming experience using Python, PySpark, Scala, Java, C#, or comparable languages.
• Experience with cloud architecture and services across Azure, AWS, GCP, or comparable cloud environments.
• Experience with APIs, distributed processing, event-driven architectures, and modern integration patterns.
• Experience with automated testing, CI/CD, source control, observability, and modern DevOps practices.
• Understanding of data governance, lineage, security, identity, and data quality practices.
• Practical experience incorporating AI-assisted tools into engineering workflows.
• Strong communication, technical judgment, problem-solving, and stakeholder-influence skills.

Nice to have

• Experience architecting solutions that span or integrate multiple data platforms or cloud technologies.
• Experience with lakehouse architectures, Apache Spark, Delta Lake, Apache Iceberg, or other open data technologies.
• Experience designing data products and applying data-as-a-product principles.
• Experience with interoperable data-sharing patterns and open data architectures.
• Experience building governed APIs or external data-delivery capabilities.
• Experience supporting AI-oriented data architectures including semantic models, RAG, data agents, machine learning, or conversational analytics.
• Experience with streaming and messaging technologies such as Kafka or comparable cloud-native services.
• Experience with API management and enterprise integration architecture.
• Familiarity with Infrastructure as Code and cloud automation.
• Experience working with sensitive or highly governed data in financial services, legal, compliance, or regulatory environments.
• Familiarity with KYC, UCC, public-record, GRC, or SOC 2 concepts.
• Experience leading data platform modernization or migration initiatives.

Details

• Location: Chennai, Neville Towers, India.
• Applicants may be required to appear onsite at a Wolters Kluwer office as part of the recruitment process.
• Interviews must be completed without AI tools or external prompts; virtual backgrounds must be removed and in-person interviews are included in the process.