Senior Data Engineer

Overview

Senior Data Engineer to design, develop, and deliver scalable enterprise data solutions across batch and real-time pipelines, governed platforms, and integration frameworks.

What you'll do

• Lead design and development of scalable batch and real-time ETL/ELT pipelines.
• Own complex data solutions from requirements and design through deployment and production support.
• Build cloud-based lakehouse and data-platform solutions using Databricks and AWS.
• Develop reusable, metadata-driven data integration frameworks for enterprise and third-party sources.
• Optimize Spark workloads, Databricks compute, SQL queries, and data storage performance and cost.
• Implement orchestration, monitoring, alerting, data-quality controls, and recovery processes.
• Implement governance capabilities including metadata management, lineage, cataloging, and RBAC/security controls.

What you'll need

• At least 7 years of relevant experience with a Master’s degree, or at least 9 years with a Bachelor’s degree.
• Advanced hands-on experience with Databricks, Apache Spark, PySpark, Spark SQL, Delta Lake, Python, and SQL.
• Experience designing and operating production-grade batch and streaming pipelines.
• Strong knowledge of distributed computing, lakehouse architecture, data warehousing, and data integration.
• Experience with Databricks Workflows (or comparable orchestration tools).
• Experience with AWS data/compute/storage/security/monitoring services.
• Experience with Git, CI/CD, automated testing, monitoring, and production deployment.

Details

• Occasional off-hours operational support is required.