Overview
Databricks Data Engineer to design, build, and maintain scalable data pipelines and data products for client AI & data projects.
What you'll do
- Design and maintain scalable data pipelines using Databricks, PySpark, SQL, and Delta Lake.
- Develop batch and streaming workflows for ingestion, transformation, validation, and publishing.
- Write and optimize Python, PySpark, and SQL code for processing, orchestration, and performance tuning.
- Troubleshoot pipeline failures, performance issues, data quality issues, and workflow bottlenecks.
- Translate requirements into data engineering designs, processing scripts, and job orchestration workflows.
- Perform data validation, quality checks, and code reviews to support reliable data products.
- Document pipelines, datasets, data models, and engineering decisions; follow governance, security, lineage, and compliance standards.
What you'll need
- 3-8 years of relevant experience in data engineering, data architecture, or cloud data platform implementation.
- Strong experience with Python, PySpark, and SQL for data transformation and pipeline development.
- Experience with Databricks, Spark, Delta Lake, Unity Catalog, or similar cloud-native data platforms.
- Experience developing batch/streaming pipelines and ETL/ELT workflows with reusable data assets.
- Experience with data modeling, data warehousing, and data quality validation for large-scale processing.
- Ability to troubleshoot technical issues and communicate recommendations clearly in team delivery environments.
- Experience applying data governance, access control, lineage, and security practices within cloud data platforms.
Read the full description and apply on the company’s own careers page.
