Overview
Research Engineer role building and evaluating the data, environments, and benchmarks used to improve advanced AI models, working closely with researchers and engineers at a frontier AI lab.
What you'll do
- Translate research goals into technical specifications, including target capabilities, task coverage, difficulty, known failure modes, and measures of success.
- Assess feasibility and define assumptions, acceptance criteria, and evaluation requirements for new work.
- Create rubrics, examples, and edge cases that make quality expectations clear.
- Review datasets and task outputs to identify errors, ambiguity, data leakage, reward exploitation, inconsistent assumptions, and gaps in coverage.
- Use quality results and failure patterns to recommend specific improvements to data creation and review.
- Design task suites, verifiers, reward functions, automated checks, and environment interfaces such as APIs and tool schemas.
- Build systems for validation, filtering, deduplication, decontamination, difficulty calibration, and diversity.
- Develop synthetic data and augmentation approaches where they are useful, such as task generators, controlled variations, and tool-driven rollouts.
- Design evaluations and analyze results to understand whether data or environments improve model performance.
- When needed, contribute to fine-tuning or reinforcement learning experiments that measure the effect of the work.
- Provide technical direction and actionable feedback to teams producing data; own technical quality and release recommendations.
- Communicate technical findings, risks, and trade-offs directly to client partners, coordinating delivery changes with the relevant project leads.
What you'll need
- Around 4–5 years of experience building or improving deep learning systems, with hands-on work in training data, post-training, evaluation, or agentic systems.
- Strong judgment about how data selection, filtering, synthesis, and evaluation affect model performance.
- Strong programming skills in Python.
- Ability to turn broad research goals into concrete technical plans and explain findings clearly to researchers and engineers.
- Careful, evidence-based approach to diagnosing data quality problems and model failure modes.
- Experience creating measurable quality processes, such as validation scripts, gold sets, sampling plans, statistical checks, or human review loops.
- Comfort working directly with clients and communicating trade-offs or setbacks.
- Ability to work independently when requirements are evolving.
Nice to have
- Experience with SQL or other structured-data workflows is helpful.
- Reinforcement learning or post-training, including RLHF/RLAIF, reward modeling, verifier training, or environment design.
- Evaluation of tool use, multi-step workflows, or long-horizon agent tasks.
- Multimodal work involving documents, charts, diagrams, OCR, or visual interfaces.
- Strong STEM background and experience checking solutions for correctness.
- Embodied AI or vision-language-action systems.
- Experience modeling application APIs, data schemas, or realistic workflow constraints.
Read the full description and apply on the company’s own careers page.