AI Benchmark Engineer | Native Language Specialist – Arabic (UAE)

Overview

Design, build, and validate multilingual Terminal-Bench tasks that evaluate coding agents and large language models on software challenges in Arabic and other localized environments.

What you'll do

• Evaluate coding agents through Terminal-Bench task engineering.
• Build realistic task environments with Arabic datasets and files.
• Identify AI failure points through native-language prompting and translation.
• Develop reference implementations and deterministic verifier scripts.
• Analyze execution logs and calibrate task difficulty across model tiers.
• Participate in human and automated quality checks for benchmark integrity.

What you'll need

• 5+ years of industry software engineering experience.
• Native or near-native Arabic fluency and high English proficiency.
• Strong proficiency in Python.
• Strong proficiency in standard shell scripting and data processing.
• Extensive Terminal/CLI-based development experience.
• Deep knowledge of multilingual text-processing issues, including Unicode and locale handling.

Details

• Freelance opportunity.
• Remote in the United Arab Emirates.
• Typical availability is at least 2 hours per day or 15–20 hours per week.