Lead Data Engineer

Overview

Lead data engineers develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions, leading the design of technical solutions and the implementation work alongside hands-on coding.

What you'll do

• Lead and manage data engineering projects from inception to completion, including goal-setting, scope definition and ensuring on-time delivery with cross-team collaboration.
• Collaborate with stakeholders to understand their strategic objectives and identify opportunities to leverage data and data quality.
• Design, develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions.
• Create, design and develop intricate data processing pipelines to address clients' most challenging problems.
• Collaborate with data scientists to design scalable implementations of their models.
• Write clean and iterative code based on TDD and leverage continuous delivery practices to deploy, support and operate data pipelines.
• Lead and advise clients on using different distributed storage and computing technologies.
• Develop data models by selecting from a variety of modeling techniques and implementing the chosen model using the appropriate technology stack.
• Be responsible for data governance, data security and data privacy to support business and compliance requirements.
• Define the strategy for, and incorporate, data quality into day-to-day work.

What you'll need

• 10+ years of experience in leading the system design and implementation of technical solutions.
• Experience creating Big Data architecture, building and operating data pipelines, and maintaining data storage within distributed systems.
• Deep understanding of data modelling and experience with modern data engineering tools and platforms.
• Experience writing clean, high-quality code using PySpark.
• Experience building and deploying large-scale data pipelines and data-centric applications using distributed storage and distributed processing platforms in a production setting.
• Experience with data visualisation techniques and communicating insights to the audience.
• Experience with data-driven approaches and applying data security and privacy strategy to solve business problems.
• Experience with different types of databases (e.g. SQL, NoSQL, data lake, data schemas).
• Understanding of stakeholder management and ability to liaise between clients and other key stakeholders, ensuring buy-in and gaining trust.
• Resilience in ambiguous situations and ability to adapt the role to approach challenges from multiple perspectives.
• Willingness to take on and skillfully manage risks and conflicts.
• Ability to coach, mentor and motivate others and influence teammates to take positive action and accountability for their work.
• Proven leadership with a track record of encouraging teammates in their professional development and relationships.
• Ability to build strong partnerships and relationships.

Details

• Hybrid work mode.
• Location: Bangalore, India.