AI Benchmark Engineer | Native Language Specialist – Chinese (Taiwan)

Overview

Design, build, and validate multilingual Terminal-Bench tasks that evaluate coding agents and large language models on software challenges in Chinese (Taiwan). This remote freelance role focuses on native-language assets, terminal workflows, and locale and encoding edge cases.

What you'll do

• Evaluate coding agents through task engineering.
• Build realistic task environments using native-language datasets and files.
• Identify AI failure points through native-language prompting and translation.
• Develop reference implementations and deterministic verifier scripts.
• Analyze execution logs and calibrate task difficulty across model tiers.
• Perform human quality checks for fairness, grammar, and benchmark integrity.

What you'll need

• 5+ years of industry software engineering experience.
• Native or near-native Chinese (Taiwan) fluency.
• High English proficiency.
• Strong proficiency in Python.
• Strong proficiency in standard shell scripting and data processing.
• Extensive Terminal/CLI-based development experience.
• Deep knowledge of multilingual text-processing issues, including Unicode and locale handling.

Details

• Remote freelance opportunity.
• Independent contractor engagement.
• Most tasks require at least 2 hours per day or 15–20 hours per week.
• Work availability fluctuates with project demand.