Data Engineering Lead

Overview

Lead data engineering for RhythmX AI's healthcare AI platform, building and managing data pipelines that extract and process data from health system EHR systems to support AI-driven healthcare applications.

What you'll do

• Develop and optimize data pipelines for the efficient extraction, transformation, loading and management of large-scale healthcare and clinical guidelines data.
• Manage sensitive healthcare data adhering to FHIR and HL7 standards, along with the version and variability of Epic Clarity extracts, Quality Measures, Drug Formularies and Clinical Guidelines, ensuring compliance with data regulations and client needs.
• Use Airflow, Temporal or other similar AI-native next generation tools to manage complex data and knowledge workflows for processing healthcare data at scale, including RAG, GraphRAG, Tool calling, MCP and other AI enabled Data Orchestration, Search, Retrieval and Aggregation.
• Support data scientists and backend developers by integrating and maintaining data systems across the organization.
• Assess and validate large-scale Gen-AI based systems using tools such as RAGAs, Opik etc.
• Apply technical skills to address unique challenges in the healthcare sector, contributing to solutions that enhance patient care.

What you'll need

• Bachelor's degree in Computer Science, Engineering, Data Science, or a related field.
• 10+ years of experience in data engineering.
• Proficiency in Python, PostgreSQL or other variant, MongoDB and Milvus Vector DB.
• Experience building and managing data pipelines using healthcare data from hospital EHR systems and other clinical data sources such as HIEs and clinical standards bodies (CMS, NCQA etc).
• Demonstrated experience with data lakes, ensuring robust and scalable data storage solutions.
• In-depth knowledge of LLM, Prompt Engineering, Embedding Techniques.
• In-depth knowledge of Prompt Optimization.
• Knowledge of various chunking strategies.
• Performance and scalability of GenAI solutions.
• Hands-on experience in Python and application to Data Engineering via Airflow.
• Hands-on Azure OpenAI stack for GPT 4, 4o, o3/o4 models.
• Hands-on experience with Weaviate/Milvus Vector DB.
• Ability to work with Onshore/Offshore team.
• In-depth knowledge of NLP.
• Guide the team for problem solving.
• In-depth knowledge in Prompt Tuning and Context Management.
• Experience in maximizing accuracy, minimizing latency, and enhancing performance of GenAI solutions.
• Taken at least one GenAI implementation into production.
• Hands-on experience with tools and frameworks like LangChain, LlamaIndex, or similar.
• Implemented at least one solution in RAG implementation.
• Strong exposure to healthcare domain data workflows.
• Hands-on experience with FHIR and HL7 standards.
• Experience in healthcare data engineering including ETL, normalization, validation, and secure data exchange.
• Ability to integrate clinical data sources (EHR/EMR systems) into AI and Analytics pipelines.
• Knowledge of tool calling, MCP, RAG/GraphRAG and other LLM based retrieval and search.

Nice to have

• Azure technology stack expertise to improve data processing and storage capabilities.
• Experience with clinical data quality management and validation.
• Knowledge of anomaly and outlier detection techniques in large datasets.
• Proficiency in querying massive datasets using database queries to drive insights and decisions.
• Experience with databases/OLAP systems such as Clickhouse and Snowflake, as well as data platforms such as Starburst.
• Experience in building and maintaining RESTful Web Services to support data integration and accessibility.
• Experience with Large Scale Recommendation Systems and managing the lifecycle of recommended items that receive end-user clinical feedback.
• Experience with learning systems (Reinforcement Learning) that use feedback as an input to Machine Learning.
• Familiarity with LLMs, RAG and fine-tuning architectures.
• Expertise in Model Monitoring and Debugging.
• Expertise in CI/CD of GenAI Solutions.
• Familiarity with HIPAA-compliant data handling and PHI/PII data governance in Healthcare AI/ML systems.
• Experience in fine tuning for Data Applications and healthcare scenarios (such as fine-tuned language models for medication terminologies).

Details

• Location: Bangalore, Karnataka.