AWS Lakehouse Data Engineer

Overview

AWS Lakehouse Data Engineer responsible for designing, implementing, and operating an AWS-native cloud data platform for AI/ML, analytics, reporting, and data visualization.

What you'll do

• Build and operate batch and streaming ingestion from APIs, relational databases, file drops, event streams, and external partners.
• Develop and optimize Python and PySpark ETL/ELT pipelines to produce analytics-ready datasets.
• Implement incremental processing, CDC, data contracts, schema validation, and reusable transformation frameworks.
• Deliver a Delta Lakehouse-style platform on Amazon S3 using Apache Iceberg and open columnar formats.
• Enable query of lakehouse data using AWS services such as Amazon Athena, Amazon EMR, AWS Glue, and Amazon Redshift.
• Set up governance, access control, metadata/catelog management, lineage, and operational data quality checks.
• Automate AWS provisioning with IaC and build CI/CD and observability for data pipelines.

What you'll need

• 6 years of relevant experience.
• Hands-on experience implementing AWS-native data lake/lakehouse architectures using Amazon S3 and services such as AWS Glue, Amazon Athena, AWS Lake Formation, and Amazon Redshift.
• Strong experience developing production ETL/ELT pipelines using Python and PySpark.
• Hands-on experience with Apache Iceberg, including ACID transactions, snapshots, schema/partition evolution, and time travel.
• Advanced SQL skills for analytical queries and data visualization workloads.
• Experience with metadata management and governance (cataloging, lineage, ownership, classification, policy enforcement, fine-grained access).
• Experience provisioning AWS resources using IaC and operating across multiple environments.

Details

• Work location: US - Remote (Any location).
• Travel required: None.