Senior Data Engineer

Overview

Senior Data Engineer in the Retail Tech team, building next-generation cloud data platforms (Snowflake, AWS) and data pipeline solutions across the bank.

What you'll do

• Contribute to the full development life cycle, including requirement analysis, functional design, technical design, programming, testing, documentation, implementation and ongoing technical support for agile projects.
• Contribute to projects/CRs utilising Snowflake, DBT, Python, PySpark, AWS cloud technologies, Ab-Initio or any ETL tools, Teradata and Oracle.
• Understand customers' core business objectives and build end-to-end data-centric solutions to address them.
• Lead or participate in the definition, execution and communication of ETL strategy and technical solution.
• Develop complex data transformations, data cleansing and data aggregation logic using Ab-Initio or any other ETL tools, DBT, PySpark and SQL.
• Ensure high-quality code by performing thorough testing, code reviews and debugging pipelines, internal framework and components, dbt models and PySpark jobs.
• Cooperate with business and operational users, other data management staff and infrastructure teams to ensure adherence to client and company guidelines, restrictions and requirements as well as industry standards.
• Improve team development related processes that accelerate delivery, drive innovation, lower cost and improve quality.
• Adhere to the Code of Conduct.

What you'll need

• Hands-on experience designing and building data solutions on Snowflake, including virtual warehouses, streams, tasks, Snowpipe, time travel and performance optimisation.
• Strong experience building modular, tested and documented data transformation models using dbt, including incremental models and CI/CD-driven workflows.
• Hands-on experience building cloud data solutions on AWS, including S3, Glue, EMR / EMR Serverless, Lambda, Step Functions / MWAA (Airflow), EventBridge, SQS, Kinesis, MSK/Kafka, Redshift, Athena, Iceberg and IAM.
• Proven experience developing scalable, distributed data processing pipelines using PySpark.
• Advanced proficiency in writing complex, performant SQL for data transformation, analysis and pipeline development.
• Solid experience with Teradata data warehousing, including utilities, query performance tuning and data modelling.
• 8+ years of experience in design or development in Analytics and Data Warehousing projects.
• Experience in designing, building, and delivering enterprise-wide data ingestion, data integration and data pipeline solutions using Big Data and Data Warehouse platforms (e.g. Teradata, Snowflake).
• Preferably at least 8 years of hands-on experience in a Data Engineering role.
• Extensive experience in ETL tools like GDE, Express-IT, Control centre, Mhub, Conduct-IT and Meta programming.
• Performance optimisation across environments, systems and resources.
• Root cause analysis: debugging complex data issues across systems.
• Implementing data quality frameworks with validation, cleansing, and reconciliation logic.
• Experience building data solutions using AWS Cloud technology (EMR, Glue, Iceberg, Kinesis, MSK/Kafka, Redshift, S3, etc.).
• Preferably 3+ years of hands-on experience with AWS and an AWS Certified Data Engineer certification.
• Strong Unix/Linux shell scripting and programming skills.
• Proficient in SQL scripting, writing complex SQLs for building data pipelines on Teradata, Snowflake and Oracle.
• Experience leading and mentoring data engineers, including ownership of internal business stakeholder relationships and working with consultants.
• Experience working in Agile teams, including working closely with internal business stakeholders.
• Experience in development, production support and troubleshooting issues.
• Experience in process optimisation (e.g. SQL, ETL).
• Experience with Big Data platforms for ETL development on the AWS cloud platform, using PySpark on EMR and Glue.
• Experience with Teradata and Oracle SQL database programming, SQL performance tuning, and relational model analysis.
• Good automation skills in areas such as data ingestion, APIs, file transfers.
• Knowledge and experience with DevSecOps practices and tools, including CI/CD for dbt models and data pipelines.
• Bachelor's degree in engineering in Computer Science/Information Technology.

Nice to have

• AWS certification (e.g. AWS Certified Data Engineer).
• Preferably extensive hands-on experience in enterprise ETL development (Ab-Initio, DataStage and etc.).
• Experience using AI tools such as Claude/Roo/Cline.

Details

• Location: Bangalore - Manyata Tech Park Road.