Arcfront logo

GenAI Researcher

Arcfront
NewPosted yesterday

LOCATION

Bengaluru · Onsite

EXPERIENCE

4 - 6 Years

TYPE

FullTime

SKILLS REQUIRED

Benchmark DesignMachine LearningStatistical ModelingData analysisLLM EvaluationPower Analysis

Job description

Overview

Own how Arcfront measures what frontier models can and cannot do in a capability area, from what gets measured, to why the measurement is valid, to what should change as a result.

What you'll do

  • Own a measurement roadmap for one capability area: what to measure now, what to retire, and what to deliberately leave for later.
  • Design new evaluations from scratch, covering construct validity, item difficulty and headroom, the power to separate the models that matter, and a contamination threat model written before the first item is authored.
  • Build agentic eval infrastructure: sandboxed environments, deterministic tool mocks, trajectory-level scoring and partial credit over long horizons.
  • Validate judges: collect human labels, measure chance-corrected agreement, run bias audits and set clear rules on where an LLM judge is and isn't trusted.
  • Apply adversarial pressure to our own numbers, hunting for reward hacking, shortcut solutions, grader exploits and prompt-format sensitivity.
  • Make the argument yourself and raise the methodological bar for the engineers around you.

What you'll need

  • 4–6 years in ML research engineering, applied research or quantitative measurement where being wrong was expensive.
  • Experience shipping measurement that other people relied on, such as an eval suite, scoring harness or experimentation platform.
  • Statistical depth in paired and clustered designs, power analysis, multiple-comparison control and calibration, including showing a team their study was underpowered with the arithmetic.
  • Systems ability in large-scale batched inference, cost and rate-limit management, containerised sandboxes and reproducibility across thousands of runs.
  • Design taste to quickly name how a proposed eval's score will stop meaning what it claims.
  • Clear, persuasive writing whose memos change decisions.

Nice to have

  • Built or substantially contributed to a public benchmark, eval harness or leaderboard.
  • Published work on evaluation methodology, contamination or judge reliability.
  • Background in psychometrics, econometrics, experimental design or causal inference applied to ML.
  • A time you proved an exciting result was an artefact and convinced the team.

Details

  • Location: Bengaluru.

Read the full description and apply on the company’s own careers page.

Stay safe

Hiring on Abekus is free for applicants

We never charge a fee, and employers are prohibited from doing so. If a recruiter asks for payment, please report them right away.

GenAI Researcher