Senior Software Engineer

Overview

Build the data and retrieval foundation that makes Clearwater's AI agents accurate, grounded, and measurable. This is a hands-on software engineering role focused on production pipelines, retrieval and evaluation systems, and the product surfaces that expose them, working with security and threat data at scale.

What you'll do

• Build and operate batch and streaming data pipelines that ingest, normalize, and enrich large volumes of security and telemetry data
• Design retrieval systems for agents: chunking, embeddings, hybrid search, reranking, vector and graph stores, and freshness and access control
• Build evaluation harnesses and datasets to measure agent and retrieval quality, regressions, and cost
• Build backend APIs and services in Python, and full-stack features that surface data and AI insights to users
• Design data models and manage relational, analytical, and vector stores for performance and cost
• Add data quality checks, lineage, observability, and governance, including PII and sensitive-data handling
• Use AI agents to automate data engineering work such as pipeline generation, data validation, and documentation
• Containerize and deploy on Kubernetes and cloud with CI/CD, and mentor engineers on data and AI practices

What you'll need

• 4+ years of professional software engineering with strong data-engineering exposure and sound testing and CI/CD practice
• 1+ year building production systems involving LLMs, retrieval (RAG), or large-scale data pipelines
• Strong Python and SQL, including query optimization and data modeling
• Hands-on experience with pipeline or stream-processing tooling such as Airflow, Dagster, Spark, or Kafka, or an equivalent
• Experience with vector stores and retrieval design (pgvector, OpenSearch, or similar), and with evaluating retrieval quality
• Docker, working Kubernetes knowledge, and experience on AWS, Azure, or GCP

Nice to have

• Working JavaScript/TypeScript, enough to build and ship full-stack features
• Experience with security, threat intelligence, or SIEM/telemetry data
• Healthcare or compliance-driven data experience (HIPAA, SOC 2, HITRUST)
• Knowledge graphs or graph databases
• LLMOps and evaluation tooling experience
• Experience with a lakehouse or warehouse such as Snowflake, Databricks, or BigQuery
• Open-source contributions or public data or retrieval projects

Details

• Location: Pune (M.H), India
• Requires a high level of initiative and independence, excellent written and oral communication skills, and the ability to work in a fast-paced environment handling multiple demands
• Requires the ability to use a personal computer for extended periods of time