IND Staff Engineer, Data

Overview

Lead data and AI engineering for a large, complex data ecosystem built on data domains, data products, cloud and a modern technology stack.

What you'll do

• Design, build and maintain scalable, robust real-time data streaming pipelines using technologies such as Apache Kafka, AWS Kinesis, Spark Streaming, or similar.
• Implement data and AI pipelines that bring together structured, semi-structured and unstructured data to support AI and Agentic solutions, including pre-processing with extraction, chunking, embedding and grounding strategies.
• Develop data-driven and AI-driven systems to improve data capabilities, ensuring compliance with industry best practices.
• Develop data domains and data products for consumption archetypes including Reporting, Data Science, AI/ML and Analytics.
• Implement efficient Retrieval-Augmented Generation (RAG) architectures and integrate them with enterprise data infrastructure.
• Collaborate with cross-functional teams to integrate solutions into operational processes and systems supporting various functions.
• Stay current with GenAI advancements, design prototypes (POCs), run experiments, and recommend innovative tools and technologies to enhance data capabilities.
• Model domain entities, relationships and business logic in knowledge graphs (e.g., Neo4j, Amazon Neptune, RDF).
• Integrate data from multiple sources, ensuring canonical representation and semantic consistency.
• Develop and validate synthetic data to simulate rare events and edge cases, and integrate synthetic data workflows with automated testing frameworks for agent evaluation.
• Identify and champion AI-driven data engineering productivity improvements, including AI-driven auto-generation of data pipelines, AI-augmented self-healing pipelines, and automated data quality frameworks.
• Design and implement a scalable semantic layer with dynamic query translation for real-time insights and conversational analytics, integrated with AI/LLM platforms under enterprise governance standards.
• Ensure reliability, availability and scalability of data pipelines and systems through monitoring, alerting and incident management.
• Implement reliability engineering best practices, including redundancy, fault tolerance and disaster recovery strategies.
• Collaborate with DevOps and infrastructure teams on deployment, operation and maintenance of data systems.
• Mentor junior team members and engage in communities of practice to promote best practices, standards and reusable patterns.
• Develop graph database solutions for complex data relationships supporting AI systems, including queries (e.g., Cypher, SPARQL) for reasoning, relationship discovery and contextual enrichment for AI agents.
• Apply GenAI solutions to insurance-specific data use cases and challenges.
• Partner with architects and stakeholders to implement the vision for AI and data pipelines while safeguarding the integrity and scalability of the environment.

What you'll need

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, or a related field.
• Data engineering experience including data solutions, SQL and NoSQL, Snowflake, ETL/ELT tools, CI/CD, big data, cloud technologies (AWS/Google/Azure), Python/Spark, data mesh, data lake or data fabric.
• Mastery-level data engineering and architecture skills, including deep expertise in data architecture patterns, data warehouse, data integration, data lakes, data domains, data products, business intelligence and cloud technology capabilities.
• Expertise with cloud platforms (AWS, GCP or Azure) and containerization technologies (Docker, Kubernetes).
• Data engineering experience focused on supporting Generative AI technologies.
• Hands-on experience with Snowflake.
• Experience building data and AI pipelines that bring together structured, semi-structured and unstructured data, including pre-processing with extraction, chunking, embedding and grounding strategies, semantic modeling, and preparing data for models and agentic solutions.
• Strong hands-on experience implementing production-ready, enterprise-grade GenAI data solutions.
• Experience with prompt engineering techniques for large language models.
• Experience implementing Retrieval-Augmented Generation (RAG) pipelines, integrating retrieval mechanisms with language models.
• Intermediate mastery in processing and leveraging unstructured data for GenAI applications.
• Intermediate mastery in implementing scalable AI-driven data systems supporting agentic solutions (AWS Lambda, S3, EC2, LangChain, LangGraph, MCP, A2A).
• Strong Python programming skills and familiarity with deep learning frameworks such as PyTorch or TensorFlow.
• Experience with vector, graph, NoSQL and document databases, including design, implementation and optimization (e.g., AWS OpenSearch, GCP Vertex AI, Neo4j, Spanner Graph, Neptune, MongoDB, DynamoDB).
• Mastery in implementing data governance practices, including data quality, lineage and data catalogue capture, on a large-scale data platform.
• Strong written and verbal communication skills, with the ability to explain technical concepts to various stakeholders.
• Expert-level collaboration, decision-making, conflict resolution and relationship-building skills.
• Expertise in mentoring and developing junior AI or data engineers.
• Familiarity with evolving industry design patterns for AI.
• Strong planning, organization and execution skills.
• Ability to provide thought leadership to dynamic, collaborative teams, with excellent interpersonal and time management skills.
• Ability to align deliverables to departmental and organizational strategies and objectives.
• Ability to lead successfully in a lean, agile, fast-paced organization using Scaled Agile principles and ways of working.
• Ability to translate complex technical topics into business solutions and strategies, and turn business requirements into technical solutions.

Nice to have

• Experience in multi-cloud hybrid AI solutions.
• Certifications in AI, GCP or Snowflake.
• Experience in the P&C or Employee Benefits insurance industry.