Data Engineer 2

Overview

Data Engineer 2 (Data Engineer P2) to build and maintain cloud-based data infrastructure and data pipelines for the Enterprise Data team.

What you'll do

• Design, develop, and maintain scalable cloud-based data infrastructure using SQL and PySpark.
• Define data ingestion architecture by partnering with cross-functional teams on data requirements and sources.
• Build and implement data pipeline frameworks to move data to data lakes, data warehouses, and analytical platforms.
• Troubleshoot data processing, data quality, and data pipeline performance issues.
• Stay current on emerging cloud data engineering tools, technologies, and best practices.
• Collaborate with data scientists and analysts to deliver data solutions that meet their needs.
• Document data infrastructure, pipelines, and ETL processes to enable knowledge transfer.

What you'll need

• Bachelor’s degree in Computer Science, Data Engineering, or a related field (Master’s preferred).
• Minimum 3-5 years of experience in data engineering or full-stack development in cloud-based environments.
• Strong big data technology expertise including Python, SQL, and PySpark/Spark.
• Strong Databricks experience.
• Ability to write complex SQL for data validation and integrity plus strong backend testing skills.
• Experience with automated tests and validation for data pipelines (including unit, functional, integration, UAT, system, and security).
• Experience with cloud platforms including AWS, Microsoft Azure, or GCP.