IND Sr Staff Engineer, Data

Overview

Senior Data and AI Engineering lead supporting Claims Invest initiatives within the Enterprise Data Services (EDS) portfolio, owning a large and complex data ecosystem built on data domains, data products, cloud and a modern technology stack.

What you'll do

• Design, build and maintain scalable and robust real-time data streaming pipelines using technologies such as Apache Kafka, AWS Kinesis, Spark streaming, or similar.
• Implement Data and AI pipelines that bring together structured, semi-structured and unstructured data to support AI and Agentic solutions, including pre-processing with extraction, chunking, embedding and grounding strategies.
• Design and develop Data and AI-driven systems to improve data capabilities, ensuring compliance with industry best practices.
• Design and develop data domains and data products for various consumption archetypes including Reporting, Data Science, AI/ML and Analytics.
• Design and implement efficient Retrieval-Augmented Generation (RAG) architectures and integrate them with enterprise data infrastructure.
• Collaborate with cross-functional teams to integrate solutions into operational processes and systems supporting various functions.
• Stay up to date with industry advancements in GenAI and apply modern technologies and methodologies, leading prototypes (POCs), conducting experiments, and recommending innovative tools to enhance data capabilities.
• Model domain entities, relationships, and business logic in knowledge graphs (e.g., Neo4j, Amazon Neptune, RDF).
• Integrate data from multiple sources, ensuring canonical representation and semantic consistency.
• Develop and validate synthetic data to simulate rare events and edge cases, and integrate synthetic data workflows with automated testing frameworks to support agent evaluation.
• Identify and champion AI-driven Data Engineering productivity improvements, including AI-driven auto-generation of data pipelines, AI-augmented self-healing data pipelines, and automated data quality frameworks.
• Design and implement a scalable semantic layer with dynamic query translation to deliver real-time insights for conversational analytics, integrated with AI/LLM platforms.
• Ensure the reliability, availability, and scalability of data pipelines and systems through effective monitoring, alerting, and incident management, and implement reliability engineering practices including redundancy, fault tolerance, and disaster recovery.
• Collaborate closely with DevOps and infrastructure teams to ensure seamless deployment, operation, and maintenance of data systems.
• Mentor junior team members and lead communities of practice to deliver high-quality data and AI solutions and promote reusable patterns.
• Design and develop graph database solutions for complex data relationships supporting AI systems, including developing and optimizing queries (e.g., Cypher, SPARQL).
• Design and apply GenAI solutions to insurance-specific data use cases and challenges.
• Partner with architects and stakeholders to influence and implement the vision of the AI and data pipelines while safeguarding the integrity and scalability of the environment.

What you'll need

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, or related field.
• 10+ years of data engineering experience including data solutions, SQL and NoSQL, Snowflake, ETL/ELT tools, CI/CD, Big Data, cloud technologies (AWS/Google/Azure), Python/Spark, Data Mesh, Data Lake or Data Fabric.
• Mastery-level data engineering and architecture skills, including deep expertise in data architecture patterns, data warehouse, data integration, data lakes, data domains, data products, business intelligence, and cloud technology capabilities.
• 5+ years expertise with cloud platforms (AWS, GCP, or Azure) and containerization technologies (Docker, Kubernetes).
• 2+ years of data engineering experience focused on supporting Generative AI technologies.
• 3+ years hands-on experience with Snowflake.
• 3+ years' experience building Data and AI pipelines that bring together structured, semi-structured and unstructured data, including pre-processing with extraction, chunking, embedding and grounding strategies, semantic modeling, and preparing data for Models and Agentic solutions.
• 2+ years strong hands-on experience implementing production-ready enterprise-grade GenAI data solutions.
• 2+ years' experience with prompt engineering techniques for large language models.
• 2+ years' experience implementing Retrieval-Augmented Generation (RAG) pipelines, integrating retrieval mechanisms with language models.
• Mastery in processing and leveraging unstructured data for GenAI applications.
• Mastery in implementing scalable AI-driven data systems supporting agentic solutions (AWS Lambda, S3, EC2, Langchain, Langgraph, MCP, A2A).
• 3+ years strong programming skills in Python and familiarity with deep learning frameworks such as PyTorch or TensorFlow.
• 2+ years' experience in vector databases, graph databases, NoSQL and document databases, including design, implementation, and optimization (e.g., AWS OpenSearch, GCP Vertex AI, Neo4j, Spanner Graph, Neptune, Mongo, DynamoDB).
• Mastery in implementing data governance practices, including Data Quality, Lineage, and Data Catalogue capture, on a large-scale data platform.
• Strong written and verbal communication skills and ability to explain technical concepts to various stakeholders.
• Expert-level collaboration skills across teams, decision making, conflict resolution and relationship building.
• Expertise in mentoring and developing junior AI or Data Engineers.
• Familiarity with evolving industry design patterns for AI.
• Strong planning, organization, and execution skills, with the ability to provide thought leadership to dynamic and collaborative teams and manage time effectively.
• Ability to understand and align deliverables to departmental and organizational strategies and objectives.
• Ability to lead successfully in a lean, agile, fast-paced organization, leveraging Scaled Agile principles and ways of working.
• Leader and team player with a transformation mindset.
• Ability to translate complex technical topics into business solutions and strategies, and turn business requirements into technical solutions.

Nice to have

• Experience in multi-cloud hybrid AI solutions.
• Certifications in AI, GCP or Snowflake.
• Experience in the P&C or Employee Benefits insurance industry.
• Knowledge of natural language processing (NLP) and computer vision technologies.
• Contributions to open-source AI projects or research publications in the field of Generative AI.

Details

• Location: Hyderabad, India.
• Employment type: Full-time.