AI Benchmark Engineer | Native Language Specialist – French (Belgium)

Overview

Native French (Belgium) engineer to design, build, and verify Terminal-Bench benchmark tasks for evaluating multilingual capabilities of large language models.

What you'll do

• Engineer and evaluate coding-agent benchmark tasks.
• Create realistic task environments using native-language datasets and files.
• Find model failure points in your native language for prompting/translation.
• Support reference implementations and write deterministic verifier scripts.
• Calibrate task difficulty using Terminal-Bench runs and analyze execution logs.
• Participate in a multi-layer human quality control process with automated checks.

What you'll need

• 5+ years of industry experience in software engineering.
• Native or near-native fluency in the native language and deep grammar/register knowledge.
• Strong proficiency in Python, standard shell scripting, and data processing.
• Experience with Terminal/CLI-based development workflows and familiarity with coding agents.
• Experience with multilingual text processing pitfalls like encoding/decoding and Unicode normalization.
• Working familiarity with locale-dependent conventions and safe string operations needed for text I/O.

Details

• Remote, freelance opportunity.