AI Benchmark Engineer | Native Language Specialist – Arabic (Saudi Arabia)

Overview

Design, build, and validate Terminal-Bench tasks that evaluate large language models on multilingual software challenges in Arabic and other non-English environments.

What you'll do

• Evaluate coding agents through terminal-based benchmark tasks.
• Build realistic Arabic-language task environments, datasets, and files.
• Identify AI failure points through native-language prompting and translation.
• Develop reference implementations and deterministic verifier scripts.
• Analyze execution logs and calibrate task difficulty across model tiers.
• Participate in human and automated quality-control reviews.

What you'll need

• At least 5 years of industry software engineering experience.
• Native or near-native Arabic fluency and high English proficiency.
• Strong Python, shell scripting, and data-processing skills.
• Extensive Terminal/CLI-based development experience.
• Working familiarity with coding agents.
• Deep knowledge of Unicode, locale, text I/O, and multilingual processing issues.

Details

• Freelance independent-contractor opportunity.
• Remote in Saudi Arabia.
• Most tasks require at least 2 hours per day or 15–20 hours per week.
• Work availability fluctuates with project demand.