MSR Project - Hindi Culture Expert

Overview

Freelance remote project to evaluate AI search responses for failures, working with Hindi-speaking experts on language evaluation.

What you'll do

• Identify realistic search-related failures where an AI model provides inadequate responses.
• Create search queries, test them, and document what went wrong.
• Write instructions and evaluation criteria for correct answers.
• Test multiple queries before finding failures.

What you'll need

• Research skills: ability to find and verify information from multiple sources, comfort navigating different websites, databases, and documentation.
• Critical thinking: can spot contradictions between sources, identify outdated information, and recognize when something doesn't apply to a specific locale or context.
• Attention to detail: can catch nuances in language, local variations (prices, regulations, requirements), and edge cases.
• Problem-solving mindset: comfortable iterating and testing when things don't work, and does not get frustrated by trial-and-error.
• Writing clarity: can articulate instructions and rubrics clearly so others understand the logic behind their reasoning.

Nice to have

• Domain knowledge: ideally a background in research, quality assurance, compliance, or similar work that requires precision and source verification.
• Locale knowledge: deep familiarity with their specific locale (local regulations, business practices, cultural nuances) is valuable for spotting region-specific failures.

Details

• Start date: ASAP.
• Duration: ongoing project.
• Commitment: 20 hours per week.
• Job type: freelance contract.
• Location: India, remote.
• Rate: $10 per hour.