Data Engineer (SDE 2)

Overview

Data Engineer (SDE 2) role on the data team to build a high-performance, scalable data lakehouse and accelerate development using AI.

What you'll do

• Develop a scalable data lakehouse targeting sub-minute data latency.
• Manage complex CDC flows.
• Optimize distributed query engines for performance.
• Accelerate engineering by leveraging AI tools for code writing, testing, and documentation.
• Drive technical initiatives through TRDs and active design reviews.
• Design OLAP domain models including facts, dimensions, cumulative structures, SCDs, and OBT pattern tables.
• Interact with product and key stakeholders to add value via data and analytics.

What you'll need

• 3–5 years of experience in data engineering with distributed systems and cloud-native architectures.
• Expert-level Python/PySpark and SQL skills.
• Hands-on AWS experience including S3, EKS, and MSK.
• Experience with Airflow or Temporal for complex workflow management.
• Experience with Infrastructure-as-Code.
• Ability to explain trade-offs between storage formats and processing frameworks.
• Proficiency using AI tools (Claude, Codex, Copilot) for code tasks.

Nice to have

• High-throughput real-time CDC ingestion from RDBMS to lakehouse using Debezium or PeerDB.
• Lakehouse table format design and optimization using Iceberg, Delta, or Hudi.
• Unified compute via ETL/ELT frameworks in PySpark and Flink for batch and streaming workloads.
• Tuning Trino or ClickHouse for real-time dashboards in Metabase, Superset, or PowerBI.
• Proficiency with Go/Java/Scala.