UserTesting logo

Solutions Consultant

UserTesting
Posted a month ago

LOCATION

Solutions Consultant R · Remote

EXPERIENCE

7+ Years

TYPE

FullTime

SKILLS REQUIRED

LLMopsModel MonitoringModel CI/CDMLOpsGoverned Model DeploymentDrift Monitoring

Job description

Overview

Sr. Engineer II, ML Ops will establish enterprise-grade AI operations (MLOps/LLMOps) and, once operational practices are team-owned, lead a transition into emerging AI research and innovation.

What you'll do

  • Own end-to-end model lifecycle design, including versioning, CI/CD, artifact registry, environment promotion, and deployment automation.
  • Implement LLMOps practices such as prompt versioning, testing harnesses, regression detection, governance, and rollback.
  • Build automated retraining (scheduled, drift-triggered, and manual intervention points) and self-healing mechanisms for degradation remediation.
  • Define evaluation, monitoring, and observability for ML and LLM systems, including dashboards, alerts, and incident response playbooks.
  • Govern AI release readiness using an AI Definition of Done, operational evidence capture, approval gates, audit trails, and safe rollout patterns.
  • Lead proof-of-concepts and convert successful experiments into production-ready capabilities for advanced/agentic AI and RAG.

What you'll need

  • 7+ years software engineering experience.
  • 5+ years leading AI/ML initiatives at production scale.
  • Deep hands-on expertise in 4+ of: MLOps/LLMOps, evaluation frameworks, drift monitoring, and CI/CD for ML and model governance.
  • Strong Python experience for writing, reviewing, and debugging production systems.
  • Experience deploying AI in regulated environments such as healthcare or fintech (or equivalent).
  • Proven ability to establish practices teams adopt and operate independently.

Details

  • Location: Bangalore, India.

Read the full description and apply on the company’s own careers page.

Stay safe

Hiring on Abekus is free for applicants

We never charge a fee, and employers are prohibited from doing so. If a recruiter asks for payment, please report them right away.

Solutions Consultant