Distinguished Engineer, Data Platforms

Overview

Distinguished Engineer for Data Platforms, focusing on migrating and delivering enterprise data pipelines on a Databricks/lakehouse-oriented platform while enabling AI-assisted engineering.

What you'll do

• Partner with the core Databricks team to plan and execute transitions of existing data pipelines to the target enterprise data platform.
• Drive repeatable engineering patterns for ingestion, transformation, testing, deployment, and monitoring across onboarded datasets.
• Design and operate cloud-native data pipelines using AWS services including S3, Glue, Lambda, Kinesis, and Lake Formation.
• Promote secure access using AWS IAM, encryption, and environment-level controls aligned with enterprise governance.
• Champion AI-assisted development tools and apply LLMs for activities like code generation, refactoring, SQL/PySpark optimization, and pipeline analysis.
• Provide technical direction for offshore onboarding teams in an asset-agnostic manner.
• Support semantic modeling and data discoverability with governed concepts, metadata, lineage, and data catalogs.

What you'll need

• 10+ years experience in data engineering, data platforms, cloud data architecture, software engineering, or related domains.
• Experience operating as a senior individual contributor/technical lead/principal/staff/distinguished engineer/architect (or equivalent) in complex data platform environments.
• Hands-on experience with modern data engineering for batch and/or streaming workflows.
• Experience with Databricks-based data platforms, including migration/transition of pipelines into lakehouse architecture.
• Strong AWS cloud-native data engineering familiarity including S3, Glue, Lambda, Lake Formation, Kinesis, and related capabilities.
• Practical experience with AI-assisted engineering tools such as Claude, GitHub Copilot, or LLMs.
• Working understanding of context engineering (schemas, constraints, data contracts, coding standards, acceptance criteria) to improve AI outputs.

Details

• Location: Hyderabad, Telangana.