Senior Principal Engineer– Data

Overview

Senior Principal Engineer – Data Engineering, AI & Healthcare Analytics to architect and lead enterprise-scale data lake platforms at GSK, a hands-on technical architect role spanning Azure cloud data engineering, metadata-driven intelligence, AI/ML frameworks, agentic AI systems and healthcare domain expertise.

What you'll do

• Lead the design and delivery of scalable Azure Services, Databricks and enterprise Data Lakehouse solutions supporting patient, commercial and healthcare data.
• Design resilient batch, incremental, CDC, API, file-based and real-time pipelines using ADF, Databricks, ADLS, Key Vault, Delta Lake, Apache Iceberg, Unity Catalog, Event Hubs and Azure integration services.
• Provide end-to-end technical leadership for bidirectional integrations between Salesforce PMP (Patient Management Services) and Data Lake, covering extraction, transformation, synchronization, reconciliation, schema evolution, error handling and downstream consumption.
• Provide hands-on technical expertise across the Salesforce ecosystem, including Apex, SOQL, REST/Bulk APIs, authentication, integration patterns and API development.
• Define scalable Salesforce integration patterns using Salesforce APIs, Bulk API, platform events, incremental processing and API throttling controls.
• Drive Databricks performance engineering through Spark optimization, cluster and job sizing, Photon, caching, file compaction, data layout optimization and workload monitoring.
• Implement secure access patterns using Unity Catalog, Microsoft Entra ID groups, RBAC, managed identities, service principals, Key Vault, encryption, column masking and row-level controls.
• Drive Infrastructure-as-Code and automated deployments using Terraform/Bicep, Databricks Asset Bundles, GitHub Actions or Azure DevOps, and environment-specific configuration management.
• Design and drive metadata-driven enterprise capabilities for data ingestion, validation, monitoring and automation, and establish reusable, configuration-driven patterns for data onboarding.
• Leverage AI/ML and LLM capabilities to enhance metadata management, anomaly detection, mapping and automated data controls.
• Enable standardized data observability, quality monitoring, alerting and operational controls that are scalable, governed, auditable and compliant.
• Use AI coding agents such as GitHub Copilot, Claude, Cursor, Devin and Factory day to day to accelerate engineering delivery.
• Design and implement Agentic AI and RAG solutions using LangChain, LangGraph, LangSmith, MCP and advanced prompt engineering.
• Build multi-agent orchestration workflows using LangGraph patterns such as supervisor-worker, specialized agents, tool calling, state management and workflow coordination.
• Design and integrate MCP servers to provide secure, governed access to enterprise tools, APIs, metadata, data platforms and business services.
• Develop RAG-based agent capabilities integrating enterprise data, metadata, vector stores and governed knowledge sources.
• Engineer agents as containerized, API-enabled microservices integrated with enterprise cloud platforms and data engineering workflows.
• Implement agent observability, tracing, evaluation, testing and monitoring using LangSmith and related enterprise capabilities.
• Establish technical controls for agent security, authentication, authorization, human-in-the-loop, auditability, error handling and responsible AI.
• Integrate and govern patient data across Salesforce, Data Lake/Lakehouse, analytics and downstream platforms, meeting data quality, security, privacy, lineage, GxP, HIPAA, GDPR and enterprise governance requirements.
• Drive technical innovation across the Data Lake and Salesforce PMP ecosystem using GenAI, Agentic AI, Conversational AI and intelligent automation.
• Develop AI agents for metadata discovery, impact analysis, schema mapping, data quality diagnostics, pipeline monitoring, failure remediation and operational support.
• Convert successful POCs into secure, governed, observable and production-ready enterprise capabilities reusable across patient and commercial data teams.
• Partner directly with Patient Management Services, Salesforce, Data Lake, product and engineering teams to translate complex patient-services requirements into production-ready technical solutions.
• Resolve complex integration challenges involving API limits, high-volume processing, schema evolution, duplicate records, retries, idempotency, reconciliation, error handling and data recovery.
• Embed production controls covering security, consent and privacy, PHI/PII protection, data quality, observability, auditability, performance and operational readiness.
• Perform end-to-end impact analysis for Salesforce PMP changes across objects, fields, Apex services, APIs, Data Lake pipelines, data models, extracts, reports and downstream consumers.

What you'll need

• 10+ years of data engineering/platform experience, including significant technical leadership at Senior/Principal Engineer level.
• Deep hands-on expertise across Azure, Databricks and enterprise Data Lakehouse platforms, including Spark/PySpark, Delta Lake, Apache Iceberg, streaming, Unity Catalog, ADF, ADLS, Key Vault and Azure integration services.
• Proven experience designing enterprise-scale, metadata-driven ingestion and engineering capabilities, including batch, incremental, CDC, API, real-time processing, data quality, observability and automated controls.
• Strong hands-on knowledge of the Salesforce ecosystem, including Apex, SOQL, REST/Bulk APIs, authentication, integration patterns and bidirectional Salesforce-to-Data Lake integrations.
• Hands-on experience building GenAI and Agentic AI solutions using MCP, LangGraph, LangChain, RAG, prompt engineering, multi-agent orchestration and related LLM technologies.
• Demonstrated day-to-day use of AI coding agents to accelerate and automate development, testing, code review, documentation, impact analysis, PR creation and engineering workflows.
• Strong understanding of patient and healthcare data, with exposure to related clinical, commercial, market access or real-world data domains.
• Experience with Infrastructure-as-Code, Git, CI/CD, automated testing, JIRA, ServiceNow and modern DevOps practices.
• Excellent technical leadership and stakeholder management, with the ability to drive complex solutions across business, Salesforce, data, platform, architecture, security and engineering teams.

Nice to have

• Advanced experience with Agentic AI architectures, including multi-agent orchestration, MCP servers, tool-enabled agents, RAG, agent observability/evaluation and Responsible AI controls.
• Experience building reusable enterprise data and AI capabilities, accelerators, APIs and engineering patterns adopted across multiple teams.
• Deep understanding of patient, pharmaceutical, commercial and real-world data models, privacy requirements and governed data-product practices.
• Experience with MLOps and AI/ML lifecycle management, including model/agent evaluation, monitoring, deployment and production operationalization.
• Familiarity with semantic models, vector databases, knowledge graphs, data contracts, metadata management and advanced enterprise integration patterns.
• Demonstrated ability to evaluate emerging technologies, lead technical POCs/FDE engagements and convert successful solutions into scalable enterprise capabilities.

Details

• Location: Bengaluru Luxor North Tower.