Overview
Data Engineer role to build and maintain data pipelines and data platform solutions for business users, working in an agile, DevOps-oriented team.
What you'll do
- Mine data from structured, semi-structured and unstructured data sources.
- Implement data flows and set up automation between various data sources.
- Help build large-scale batch and real-time data pipelines with data processing frameworks like PySpark and Spark.
- Ensure data quality, interpret and analyse data problems.
- Develop, construct, test and maintain pipelines.
- Maintain data documentation and definitions.
- Perform feasibility studies to drive business users toward the best technical solutions.
- Contribute to functional analysis with business users and translate user needs into structured data pipelines.
- Prepare data for predictive and prescriptive modelling and develop data set processes.
- Elaborate solution mock-ups in an agile manner together with business users.
- Develop and test each solution component toward its industrialization.
- Contribute to the evolution of the data platform, elaborate system improvements, design innovative solutions and define best practices to keep the platform sustainable.
- Work with Agile and DevOps techniques to deliver solutions in a timely manner.
- Keep track of new tools and potentially obsolete ones through a technological watch.
- Contribute to use case industrialization initiatives and action plans.
- Share expertise with team members to facilitate knowledge transfer and lead internal investment projects.
- Communicate across departments and understand what drives the requirements of functional leaders and the business team.
- Collaborate with others, considering their perspectives on methodologies and technologies.
What you'll need
- 3 to 5 years of experience.
Nice to have
- Bachelor's or master's degree in computer science engineering.
- Experience in Big Data solutions, with wide knowledge of BI and Big Data tools, and IT and cloud environments.
- Hands-on experience with PySpark/Python.
- Good knowledge of PL/SQL/SQL.
- Knowledge of Agile software development methodologies.
- Curious and creative mind, able to turn data into compelling storylines.
- Excellent oral and written communication skills.
- Excellent problem solving and troubleshooting skills.
- Process oriented with great documentation skills.
- Fluent English (verbal and written).
Details
Read the full description and apply on the company’s own careers page.