Data Engineer - Enterprise Data Lakehouse & Data Products

Overview

Design and deliver scalable data pipelines and data products on 3M's AWS-based Enterprise Data Lakehouse, following the Bronze → Silver → Gold Medallion architecture.

What you'll do

• Lead the design and delivery of scalable data pipelines and data products on 3M's AWS-based Enterprise Data Lakehouse, using Databricks, PySpark, Python and SQL, following the Bronze → Silver → Gold Medallion architecture.
• Implement data quality, validation, reconciliation and monitoring controls, along with RBAC and data masking, including SOX-compliant controls for close-critical financial data.
• Determine data needs by partnering with analysts, product owners and architects across Finance, Supply Chain, R&D, Tax and Commercial, and turn them into reusable, analytics-ready datasets through the POC → Pilot → Prod pathway.
• Support the platform and consumption layers, including AWS services (S3, Glue, Step Functions, Lambda, Athena), Apache Iceberg, Databricks, Snowflake objects, CI/CD, and Power BI and semantic models.

What you'll need

• Bachelor's degree or higher in Computer Science, Information Technology, Engineering or a related discipline (completed and verified prior to start).
• 5+ years of experience in data engineering or analytics engineering in a private, public, government or military environment.
• Hands-on experience with SQL and relational databases, plus Python and PySpark for data processing and transformation.
• Experience with Databricks and Lakehouse/Medallion architecture, and with AWS data services (S3, Glue, Lambda, Athena or equivalent).
• Experience developing and supporting production data pipelines using Git and CI/CD practices, with an understanding of data modeling, data quality, validation and monitoring.

Nice to have

• Experience with data build tool for SQL-based transformation and testing, Snowflake as a SQL consumption layer, and Apache Iceberg or Delta Lake.
• Experience with Databricks Unity Catalog and orchestration tools (Step Functions, Airflow), and knowledge of data contracts, observability, SOX-compliant controls and catalog tools (Collibra, Alation, Glue Data Catalog).
• Relevant Databricks, AWS, Snowflake or data build tool certifications.
• Clear communication, analytical problem-solving, and collaborating with business stakeholders in Agile, data-product-oriented teams.

Details

• Location: Bangalore (Hebbal).
• Work mode: Hybrid.