Senior Data Engineer

Overview

Develop, test and implement data engineering solutions to generate analytical and reporting outputs, prepare data for data science outcomes, and manage metadata structures.

What you'll do

• Design and build automations and incorporate low code-no code solutions to accelerate development cycle time.
• Leverage technical data dictionaries and business glossaries to analyze the datasets.
• Perform data profiling and data analysis for source systems and target data repositories.
• Understand metadata and the underlying data structures needed to standardize data load processes.
• Develop data mapping specifications based on data analysis results and functional requirements.
• Perform a variety of data loads and data transformations using multiple tools and technologies.
• Build automated Extract, Transform & Load (ETL) jobs based on data mapping specifications.
• Validate data mapping results and match them with expected results.
• Leverage/Implement Data Quality (DQ) rules provided.
• Provide necessary support for post-deployment related activities when needed.

What you'll need

• Bachelor's Degree in computer science or equivalent with minimum 5+ years of hands-on experience.
• Ability to understand logical and physical data models, big data storage architecture, data modeling methodologies, metadata management, master data management and data lineage techniques.
• Hands-on experience in AWS services, especially Redshift, Glue, RDS, Step Functions, S3, SageMaker, Kinesis.
• Hands-on experience in programming languages like Python and Unix.
• Hands-on experience in writing SQL scripts for Oracle, MySQL, PostgreSQL.
• Experience handling both Online Transaction Processing (OLTP) and Online Analytical Processing (OLAP) data models.
• Experience with data warehousing and Big Data, NRT - Kafka.
• Exposure to unstructured datasets and ability to handle XML and JSON file formats.
• Exposure to Extract, Transform & Load (ETL) tools like Matillion or Informatica.
• Exposure or experience in Databricks.
• Experience using 'Low code, No code' tools like Microsoft/GitHub Copilot.
• Exposure to handling enterprise ERP and CRM data.
• Knowledge of industrial applications in commercial, finance or manufacturing settings.
• Exposure to finance and accounting data domains.
• Partner with other team members to understand project objectives and resolve technical issues.
• Communicate project status or challenges clearly and concisely to cross-team members.
• Ability to drive technical solutions to meet business outcomes.

Details

• Location: Bengaluru, India (EPIP 122, Phase II).