Data Engineer

Overview

Design, build, maintain and analyze data pipelines and data solutions that deliver reliable, accessible data and actionable insights to drive business decisions.

What you'll do

• Design, develop and maintain data solutions for data generation, collection and processing
• Assist in the design and development of data pipelines
• Build data pipelines and ensure data quality by implementing ETL processes to migrate and deploy data across systems
• Contribute to the design, development and implementation of data pipelines, ETL/ELT processes and data integration solutions
• Design, develop and implement AI solutions using Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), vector search, tokenization, prompt engineering and AI-assisted development tools such as OpenAI Codex, GitHub Copilot, or equivalent platform
• Take ownership of data pipeline projects from inception to deployment, managing scope, timelines and risks
• Collaborate with cross-functional teams to understand data requirements and design solutions that meet business needs
• Develop and maintain data models, data dictionaries and other documentation to ensure data accuracy and consistency
• Implement data security and privacy measures to protect sensitive data
• Leverage cloud platforms (AWS preferred) to build scalable and efficient data solutions
• Collaborate and communicate effectively with product teams
• Collaborate with Data Architects, Business SMEs and Data Scientists to design and develop end-to-end data pipelines to meet fast-paced business needs across geographic regions
• Identify and resolve complex data-related challenges
• Adhere to best practices for coding, testing and designing reusable code and components
• Explore new tools and technologies that will help improve ETL platform performance
• Participate in sprint planning meetings and provide estimations on technical implementation

What you'll need

• Master's or Bachelor's degree and 5 - 9 years of Computer Science, IT or related field experience
• Hands-on experience with big data technologies and platforms, such as Databricks, Apache Spark (PySpark, SparkSQL), workflow orchestration and performance tuning on big data processing
• Proficiency in data analysis tools (e.g. SQL) and experience with data visualization tools
• Excellent problem-solving skills and the ability to work with large, complex datasets
• Solid understanding of data governance frameworks, tools and best practices
• Knowledge of data protection regulations and compliance requirements
• Excellent critical-thinking and problem-solving skills
• Good communication and collaboration skills
• Demonstrated awareness of how to function in a team setting
• Demonstrated presentation skills

Nice to have

• Experience with ETL tools such as Apache Spark, and various Python packages related to data processing and machine learning model development
• Good understanding of data modeling, data warehousing and data integration concepts
• Knowledge of Python/R, Databricks, SageMaker and cloud data platforms
• Hands-on experience designing, developing and implementing AI solutions using Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), vector search, tokenization, prompt engineering and AI-assisted development tools such as OpenAI Codex, GitHub Copilot, or equivalent platform
• Certified Data Engineer / Data Analyst (preferred on Databricks or cloud environments)