Senior Associate, Data Engineer

Overview

The Senior Associate, Data Engineer role focuses on designing, building, measuring, and supporting enterprise data solutions, including ELT jobs, data pipelines, integrations, transformations, and BI solutions.

What you'll do

• Design, build, and measure ELT jobs to process disparate data sources and form a high integrity, high quality, clean data asset.
• Work closely with tech leads and senior engineers to implement and deliver enterprise data solutions, including data strategy, solution feasibility and longevity, hands-on implementation, and BAU support.
• Involve stakeholders in conversations to define multi-market data requirements and implement data and BI solutions.
• Create and maintain data pipelines, integrations, and transformations to ensure efficient data processing, storage, and retrieval.
• Ensure data infrastructure is scalable, efficient, and aligned with best practices in data management.
• Extract, organize, and manage data to ensure accessibility for authorized users across the organization.
• Perform data profiling, validation, and cleansing to maintain data integrity and accuracy.
• Perform unit tests and conduct reviews with other team members to ensure code is rigorously designed, elegantly coded, and effectively tuned for performance.
• Implement internal process improvements, automate manual processes, and debug inefficient pipelines for optimal performance and scalability.
• Follow best practices and proactively seek opportunities to automate and construct tools that streamline operations and enhance overall reliability.
• Collaborate with business analysts to understand requirements and use cases, translating them into technical stories and tasks.

What you'll need

• 3-5 years of experience in Data Warehousing and Business Intelligence roles.
• Proficiency in Airflow Orchestration Pipelines, relational database design, data modelling, and data warehousing principles.
• Excellent programming skills in SQL and Python.
• Proficiency in Shell scripting and building Spark jobs.
• Knowledge of AWS services such as S3, Lambda, EMR, CloudWatch, and Step Functions.
• Knowledge of cloud services, relational database design principles, data warehousing, performance tuning, and optimization techniques.
• Experience in performance analysis and tuning to ensure optimal database performance.
• Ability to analyze business requirements.
• Excellent communication skills with the ability to collaborate effectively with technical teams and business stakeholders to gather and refine requirements.
• Delivery-focused with a can-do attitude in challenging environments.

Details

• Location: Hyderabad, India.