AI Architect

Overview

Principal AI Architect (individual contributor, hands-on) who owns the architecture underneath every FourKites AI agent, including the harness, context strategy, model choices and evaluation framework, and sets its direction across agent teams. Based in Chennai (hybrid).

What you'll do

• Define the reference architecture for FourKites agents: the agent loop, tool execution, state and checkpointing, retries and recovery, guardrails, permissions, and human-in-the-loop handoffs.
• Design for long-running, multi-step work that spans many context windows and recovers cleanly when a tool, carrier API or model call fails.
• Decide where each problem is best solved (frontier LLM, open-weight model, fine-tuned encoder, or plain code) and own model routing, cost-per-task and latency budgets across the fleet.
• Integrate agents with FourKites' event-driven platform (Kafka, microservices, Postgres) and with customer systems through APIs and MCP servers.
• Set the standard for what goes into an agent's context window (system prompts, tool definitions, retrieved documents, memory, prior turns) within an explicit token budget.
• Build shared components for retrieval, compaction and summarization, sub-agent isolation, prompt caching and persistent memory.
• Diagnose context failures (lost instructions, tool confusion, quality degradation as the window fills) and turn fixes into reusable harness patterns.
• Build the offline eval framework: golden datasets from production traces, code-based checks, LLM-as-judge graders calibrated against human labels, and CI regression suites that gate every prompt, model or tool change.
• Build online evaluation: scoring sampled production traffic, trace- and task-level quality metrics, drift detection, shadow and A/B rollouts, user feedback signals, and quality alerts.
• Connect eval metrics to the business outcomes customers pay for, such as tracking accuracy, exceptions resolved without a human, and appointment fill rate.
• Review designs and PRs for agent teams, write the architecture docs and playbooks they follow, and mentor staff and senior engineers.
• Track the model and tooling landscape, run the experiments that matter, and give leadership clear build, buy or wait recommendations.
• Partner with Product, Solutions and Customer Success to turn field failures into platform fixes.

What you'll need

• Experience architecting and operating at least one LLM agent system with real users and real traffic (tool calling, multi-step workflows, failure handling, human escalation), owned through incidents, cost and latency after launch.
• Deep ML knowledge of encoders and transformers: attention, positional encodings, encoder-only vs decoder-only vs encoder-decoder, tokenization and the KV cache.
• Trained or fine-tuned encoder models (BERT-family, sentence embeddings, bi-encoders, cross-encoder re-rankers) for classification, extraction or retrieval, and an understanding of how contrastive training, distillation and quantization change cost and quality.
• Context engineering experience: managing token budgets, building retrieval and memory for agents, using compaction and sub-agents for long tasks, and measuring how quality changes as the window fills.
• Harness engineering experience: building the agent loop, tool dispatch, state and checkpoints, interruption and recovery, sandboxing and permissions, guardrails and model routing; fluent with MCP, structured outputs and streaming.
• Hands-on online and offline evals: datasets, graders, LLM-as-judge calibration and CI regression gates offline; production scoring, trace sampling, drift monitoring, shadow deployments and feedback loops online; has used evals to block a release or justify a model switch.
• 10+ years building software or ML systems, with 3+ years building on LLMs.
• Strong, hands-on Python; comfortable reading and reviewing Java or TypeScript services.
• Production experience on AWS or GCP, containers and Kubernetes, and event-driven systems.
• Excellent oral and written communication: able to explain a trade-off to an engineer and its business impact to an executive.

Nice to have

• Experience in supply chain, logistics or transportation.
• Document AI: OCR, layout models, and extraction from bills of lading, invoices and PODs.
• Voice or email agents that act on behalf of a business.
• Agent security: prompt injection defence, tool permissioning, and tenant data isolation.
• Hands-on experience with several model families (Claude, GPT, Gemini, open-weight models) and serving stacks.
• Published work, talks or open-source contributions on agents, retrieval or evaluation.

Details

• Location: Chennai (hybrid).
• Individual contributor role: writes code most weeks and leads design reviews across agent teams.
• Works closely with agent teams in Chennai and with colleagues in the US and Europe.