Applied AI Engineer

Overview

Applied AI Engineer on the platform engineering team, owning how mpathic's production services are built on large language models. Sits with the CTO's platform team and works closely with the AI/ML team, research, and Security and Compliance.

What you'll do

• Own the LLM gateway: design, build, and run the single path through which production services call model providers, with logging, cost attribution, rate limiting, failover, and PHI controls.
• Remove direct provider API keys from services so nothing reaches a model except through the gateway.
• Own the approved model catalog of models, providers, endpoints, and model features, documenting data retention, BAA coverage, and version pinning for each, and recording additions in short decision records.
• Build the release path for model and prompt changes: version prompts and model configurations alongside code, run the AI/ML team's evals as CI release gates, and support staged rollout and rollback.
• Give engineers and researchers self-service templates so sandbox and staging move fast while production stays gated.
• Productize AI/ML prototypes and their eval harnesses and acceptance thresholds into scalable, observable production services.
• Advise the AI/ML team on the cost, latency, and availability of the models they choose.
• Instrument LLM services end to end with OpenTelemetry (latency, failures, token usage, cost, output quality) and monitor for drift against the AI/ML team's evals.
• Join on-call and lead incident response for LLM-backed services.
• Partner with Security and Compliance to produce audit evidence for SOC 2, ISO 27001, HIPAA, and FDA 21 CFR Part 11.

What you'll need

• 5+ years of backend software engineering, including multi-tenant SaaS services and APIs in production.
• Production experience building on commercial LLM APIs (for example Anthropic, OpenAI, AWS Bedrock, or Azure OpenAI), including prompt management, structured outputs, and tool use.
• Hands-on experience designing evaluation pipelines for LLM output and wiring them into CI/CD as release gates.
• Strong Python.
• AWS, containers (ECS or similar), and infrastructure as code (Pulumi, Terraform, or CDK).
• Observability tooling such as OpenTelemetry, Datadog, or similar, and the delivery toolchain (version control, feature flags, CI, and an issue tracker such as Linear).
• Working knowledge of application security: secrets management, access control, and handling of sensitive data.

Nice to have

• NodeJS.
• Experience in regulated domains such as healthcare or life sciences, including HIPAA, SOC 2, ISO 27001, or FDA 21 CFR Part 11 validated systems.
• Experience building an internal developer platform or paved road used by other teams.
• Experience with LLM-as-judge evaluation, red teaming, or conversational analytics.
• Experience with training or fine-tuning models from scratch.