D

Applied AI Engineer

DistantJob
NewPosted yesterday

LOCATION

Remote

EXPERIENCE

Not specified

SKILLS REQUIRED

Prompt EngineeringLarge Language ModelsPythonModel EvaluationRAGAgent orchestration

Job description

Overview

Design, build and test the AI conversations and agents inside the client's learner-facing products, including advisory experiences, practice simulations and diagnostic coaching.

What you'll do

  • Turn scenario specs, decision models and assessment rubrics into prompts, application logic and testable behavior.
  • Spot missing requirements and technical constraints, and raise them with Product, Engineering and SMEs.
  • Define how AI models connect to application services and approved data sources: APIs, tool interactions, data structures, validation, error handling, permitted actions.
  • Design prompts, structured outputs and validation logic, and catch invalid responses before they reach learners.
  • Build conversational runtimes with controlled personas, session state and explicit state transitions.
  • Keep simulation behavior, assessment logic and feedback separate, so feedback cannot change assessment results.
  • Implement grounding and retrieval from approved knowledge sources, with source references where required.
  • Keep code, prompts, schemas and evaluation assets under version control, and document them.
  • Run bounded experiments with Product, with clear questions and success measures.
  • Take validated prototypes into production with Engineering, and check behavior stays the same.
  • Help build the technical controls that keep AI features compliant with accreditation requirements (issue flagging, review steps, disclosure, logging).

What you'll need

  • Expert prompt and context engineering: system prompts, few-shot examples, prompt chaining, routing, breaking complex tasks into steps.
  • Structured outputs with JSON Schema, function calling and validation layers, with retry and fallback logic.
  • RAG design: chunking, metadata filtering, hybrid search, reranking, citation grounding.
  • Managing token budgets, prompt caching, context compaction and long-session memory.
  • Agentic systems: tool use, planner/executor patterns, multi-agent orchestration, state machines, human-in-the-loop checkpoints.
  • Defining tool schemas, permission scopes and action limits for agents.
  • AI evaluation: golden datasets, rubric-based scoring, LLM-as-judge calibrated against human raters, regression testing.
  • LLM observability and tracing.
  • LLM security: prompt injection, data leakage, jailbreaks, unsafe tool actions, and guardrails.
  • Comparing prompting, retrieval and model options on quality, latency and cost.
  • Python or TypeScript, including API integrations and automated testing.
  • Git (e.g. Bitbucket), reproducible tests and CI.
  • Works with AI coding agents by default.
  • Explains technical trade-offs to non-technical colleagues.
  • Sound judgment on privacy, human oversight and the limits of AI-generated assessment or advice.
  • Hands-on experience building LLM applications with multi-step conversational workflows.
  • Experience collaborating with subject-matter experts.

Nice to have

  • Background in learning products, assessment, simulations or regulated professional content.

Details

  • Fully remote, long-term opportunity.

Read the full description and apply on the company’s own careers page.

Stay safe

Hiring on Abekus is free for applicants

We never charge a fee, and employers are prohibited from doing so. If a recruiter asks for payment, please report them right away.

Applied AI Engineer