Data Engineer II - Snowflake and DBT

Overview

Design, build and maintain enterprise data pipelines and data platforms supporting the bank's data warehouse, data lake and cloud solutions.

What you'll do

• Build and maintain data platforms supporting integrations for the Enterprise Data Warehouse, Operational Data Store or Data Marts, with appropriate data access, security, privacy and governance.
• Create data ingestion pipelines in data warehouses and large-scale data platforms.
• Create data ingestion pipelines for a variety of sources: files (flat, delimited, Excel), databases, APIs (with Apigee integration) and SharePoint.
• Build reusable data pipelines and frameworks using Python.
• Create scheduled and trigger-based ingestion patterns using scheduling tools.
• Create performance-optimized DDLs as per the Logical Data Model.
• Performance tune complex data pipelines and SQL queries.
• Perform impact analysis of proposed changes on existing architecture, capabilities, system priorities and technology solutions.
• Work in an Agile framework, participating in agile ceremonies and coordinating with scrum master, tech lead and product owner on sprint planning, backlog creation, refinement, demos and retrospectives.
• Work with Product Owners on PI goals, PI planning, requirement clarification and delivery coordination.
• Provide technical support for production incidents and failures.
• Work with global technology teams across time zones (primarily US) to deliver timely business value.

What you'll need

• 5+ years of experience building data pipelines in data warehouse and data lake platforms.
• Good knowledge of data warehouse principles and concepts.
• Strong knowledge of the Snowflake data platform and solutions for enterprise-level implementation and migration from legacy or on-prem systems.
• Technical expertise in AWS cloud services such as S3, Lambda, Glue, Step Functions, CloudWatch, etc.
• Experience designing and developing complex data pipelines.
• Technical expertise in ELT tools such as DBT, Qlik Replicate, Snowpark, etc.
• Expert in SQL.
• Experience with Unix and Python scripting.
• Experience in data pipeline orchestration using Airflow.
• Knowledge of logical and physical data modeling using relational or dimensional modeling practices, and high-volume ETL/ELT processes.
• Performance tuning of data pipelines and database objects to deliver optimal performance.
• Excellent verbal, written and interpersonal communication skills.
• Excellent problem-solving skills, incident management, root cause analysis and proactive solutions to improve quality.
• Ability to develop and maintain partnerships with business and IT stakeholders.
• Attention to detail, ensuring accuracy and thoroughness in all tasks.
• Ability to support peers, team and department management as a team player.

Nice to have

• Data Vault 2.0 experience.
• Experience with GitLab version control and CI/CD processes.
• Experience working in the financial industry.

Details

• Location: Bangalore, India.
• Work mode: Hybrid.
• Career level: P2; job category: Senior Associate.
• Reports to: Senior Manager, Data Engineering.