AI Benchmark Engineer | Native Language Specialist – Spanish (Mexico)

Overview

AI Benchmark Engineer (Spanish - Mexico) to design, build, and validate a verifiable evaluation suite of Terminal-Bench tasks for multilingual LLM software challenges.

What you'll do

• Engineer and evaluate coding-agent benchmark tasks.
• Create realistic task environments using datasets and files in Spanish (kept in the target language).
• Identify model failure points in your native language.
• Support robust reference solutions and write deterministic verifier scripts.
• Analyze execution logs and calibrate task difficulty using Terminal-Bench run configurations.
• Participate in a 4-layer human quality control process with automated checks.

What you'll need

• 5+ years of industry experience in software engineering.
• Native or near-native fluency in Spanish with deep understanding of grammar, register, and phrasing.
• High English proficiency.
• Strong proficiency in Python, standard shell scripting, and data processing.
• Experience with Terminal/CLI-based development workflows and familiarity with coding agents.

Details

• Remote freelance/independent contractor from Mexico (Spanish - Mexico).
• Most tasks require a minimum of 2 hours per day or 10 hours per week.