AI Benchmark Engineer | Native Language Specialist – Chinese (Hong Kong)

Overview

Design, build, and validate multilingual Terminal-Bench tasks that evaluate coding agents and large language models in Chinese (Hong Kong) software environments.

What you'll do

• Evaluate coding agents through Terminal-Bench task engineering.
• Build realistic task environments with datasets and files in Chinese.
• Identify AI failure points through native-language prompting and translation.
• Develop reference implementations and deterministic verifier scripts.
• Analyze execution logs and calibrate task difficulty across model tiers.
• Participate in human reviews and automated quality checks.

What you'll need

• Have 5+ years of industry software engineering experience.
• Demonstrate native or near-native Chinese fluency and high English proficiency.
• Use Python, standard shell scripting, and data processing.
• Have extensive Terminal/CLI-based development experience.
• Understand multilingual text-processing issues including encoding, Unicode normalization, and locale conventions.
• Understand text I/O, toolchain interoperability, and safe string operations.

Details

• Work remotely from Hong Kong.
• Work as an independent contractor on a freelance basis.
• Most tasks require at least 2 hours per day or 15–20 hours per week.