Lead Engineer - Data

Overview

Lead data engineering projects at M&G India, building and operationalizing data solutions for the business on Azure, and leading a team of data engineers.

What you'll do

• Work collaboratively with colleagues and take personal accountability to maintain and enhance controls you are responsible for, to support improvement of the overall control environment, customer outcomes and a reduction in M&G plc's operational risk.
• Lead data engineering projects to build and operationalize data solutions for the business using Azure services in combination with custom solutions, including Azure Data Factory, Azure Data Flows, Azure Databricks, Azure Data Lake Gen 2 and Azure SQL.
• Provide technical guidance to a team of data engineers and ensure alignment with agreed architectural principles.
• Design and implement data engineering, ingestion and transformation functions using ADF and Databricks.
• Migrate on-premise data warehouses to data platforms on Azure cloud.
• Implement Lakehouse/Data Warehouse solutions using modern data platform architecture.
• Carry out capacity planning and performance tuning on ADF and Databricks pipelines.
• Work in secured Azure environments using Azure KeyVaults, Service Principals and Managed Identities.
• Work with offshore, office-based development teams, collaborating within a team environment and participating in project lifecycle activities such as requirement analysis, testing and release.
• Manage stakeholders, maintain process adherence, and handle planning and documentation.
• Interact with the business for requirement gathering and query resolution.

What you'll need

• Proven experience leading a team of data engineers.
• Experience designing and implementing data engineering, ingestion and transformation functions using ADF and Databricks.
• Experience migrating on-premise data warehouses to data platforms on Azure cloud.
• Experience in Lakehouse/Data Warehouse implementation using modern data platform architecture.
• Experience of capacity planning and performance tuning on ADF and Databricks pipelines.
• Experience working in secured Azure environments using Azure KeyVaults, Service Principals and Managed Identities.
• Experience working in offshore, office-based development teams.
• Stakeholder management, process adherence, planning and documentation skills.
• Ability to interact with the business for requirement gathering and query resolution.

Details

• Location: Pune.