Senior Engineer, Agent Harness Engineering

Overview

Senior Engineer on Intellectsoft's agent harness team, building and deploying AI agent systems for pharmaceutical and life sciences clients, split roughly evenly between internal research and client delivery.

What you'll do

• Design and build core agent harnesses, covering function calling, tool interfaces, context assembly, retrieval integration, subagent delegation, retry logic, and failure handling.
• Take agentic systems from initial concept to scalable production, managing deployment, monitoring, cost, latency, and operational support against live enterprise client data.
• Build and maintain rigorous evaluation suites, task benchmarks, scoring rubrics, and regression tests using tools like Langfuse, Pydantic Logfire, or custom SDKs to drive architectural decisions.
• Integrate client enterprise systems, data sources, and internal APIs as accessible agent tools, including building and managing Model Context Protocol (MCP) servers.
• Translate ambiguous client requirements into robust agent architectures, clearly advising stakeholders when an agentic pattern is or is not the right solution.
• Interface directly with client stakeholders and articulate technical trade-offs clearly to non-technical audiences.
• Contribute reusable harness components to shared internal tooling and document design decisions to meet audit, validation, and handover standards.

What you'll need

• 3 years of professional software engineering experience, including substantive work building LLM-based or agentic systems (beyond prompt-level experimentation).
• Proven track record of deploying user-facing AI applications to production, including operational experience with monitoring, versioning, rate limiting, cost control, and incident response.
• Strong Python skills with production-grade test coverage, typing, asyncio, Pydantic, Pytest, packaging, version control, CI/CD, and robust API design.
• Hands-on experience with LLM APIs, function/tool calling, structured outputs, streaming, and context window management.
• Experience building or extending orchestration frameworks (such as LangGraph, deepagents, OpenAI Agents SDK, or custom harnesses) with a clear view on framework utility.
• Ability to define success criteria, design evaluation datasets, and systematically evaluate non-deterministic AI outputs.
• Working knowledge of containerization and at least one major cloud provider to ship and maintain services independently.
• Strong written and verbal communication skills with the ability to represent technical positions credibly and thrive amid shifting project priorities.

Nice to have

• Prior exposure to healthcare, pharma, or biotech data (clinical trial data, real-world evidence, regulatory submissions, pharmacovigilance, or commercial analytics) and standards (CDISC, SNOMED, MedDRA).
• Familiarity with GxP, 21 CFR Part 11, HIPAA, GDPR, audit logging, and formal system validation.
• Hands-on experience with LLM tracing tools (LangSmith, Langfuse, Braintrust, OpenTelemetry) or developing and hosting MCP servers.
• Experience with RAG pipelines, vector search (pgvector, Qdrant, Weaviate), fine-tuning, distillation, or model serving (vLLM, Bedrock, Vertex AI).
• Infrastructure as Code (Terraform, GitHub Actions, AWS/Azure) alongside large-scale data processing using Polars, Pandas, DuckDB, and advanced SQL.
• Prior consulting/client services experience, active use of agentic developer tooling (such as Claude Code), or open-source contributions/publications in agent architectures and evaluations.

Details

• Location: Bengaluru, Karnataka, India; Pune, Maharashtra, India.
• Flexible hours and work setup.