Quality Analyst-AI Voice Call-QC

Overview

Own the quality of AI voice agent conversations across merchant-facing campaigns by auditing calls and transcripts, refining prompts, identifying compliance and conversation gaps, and validating quality improvements in production.

What you'll do

• Listen to AI voice call recordings and review transcripts daily against prompts, conversation flows, business rules, and expected outcomes.
• Identify incorrect or off-script responses, hallucinations, repetition, long monologues, missed objections, incorrect intent identification, wrong language or script handling, interruptions, unnecessary probing, and poor call closure.
• Write, test, and refine production prompts covering conversation flow, objection handling, edge cases, language handling, tool and variable usage, and closure logic.
• Classify issues by root cause, including prompt gaps, business logic, STT/TTS errors, latency, tool/API failures, and data issues, and route them to the right owner.
• Test new and updated AI agents before go-live with test calls and scenario-based cases covering happy paths, objections, edge cases, and negative scenarios.
• Validate every prompt change on live calls and check for regressions elsewhere in the flow.
• Define and maintain QC parameters, audit checklists, scoring rubrics, and critical and non-critical error categories.
• Flag critical compliance gaps, including mis-selling, incorrect financial information, unauthorized promises, disclosure misses, and data privacy lapses, and ensure immediate corrective action.
• Verify that post-call dispositions, feedback tags, and outcome capture accurately reflect each call.
• Publish regular quality reports and dashboards covering quality scores, issue categories, trends, fixes shipped, and impact.
• Share call-level insights with Campaign Managers, Product, and Business teams and convert recurring issues into prompt, flow, or platform improvements.
• Audit Hindi, English, Hinglish, and regional-language calls for accuracy, tone, pronunciation, and language-switching behaviour.
• Maintain audit logs, prompt change history, test cases, known issues, and quality benchmarks for every campaign.

What you'll need

• 2–5 years of experience in Quality Assurance, Call Quality Audits, Contact Centre Quality, Conversational AI, or a related domain.
• Hands-on experience auditing voice calls or chat conversations against defined quality parameters and scorecards.
• Hands-on experience writing and refining prompts for AI/LLM-based applications, preferably voice agents.
• Ability to analyze AI conversations and pinpoint issues related to intent understanding, prompt adherence, hallucination, repetition, tone, compliance, and outcome achievement.
• Strong listening skills and the ability to catch subtle errors in AI responses, language, and flow.
• Strong analytical and problem-solving skills with proficiency in Excel/Google Sheets.
• Fluency in Hindi and English.
• Excellent written and verbal communication skills with strong documentation capabilities.
• Strong ownership and ability to audit multiple AI campaigns in parallel with fast turnaround.

Nice to have

• Proficiency in one or more regional languages.
• Experience in FinTech, Payments, Lending, Merchant Operations, or Contact Centre Operations.
• Hands-on experience with AI voice agents, OBD, or IVR quality audits.
• Understanding of LLMs, speech-to-text, text-to-speech, intent classification, tool calling, and agent workflows.
• Familiarity with templating in prompts, such as Jinja2, and structured prompt formats.
• Experience using LLMs to automate transcript analysis and QC at scale.
• Exposure to SQL, dashboards, and CRM/ticketing systems.

Details

• Location: Noida, Uttar Pradesh.