Overview
Platform engineer on the Human Data Division's platform team, maintaining and extending the company's evaluation, RL and annotation platforms that are already in use by frontier AI labs.
What you'll do
- Maintain and improve the platforms: learn the existing codebases in depth, fix bugs, pay down technical debt, improve test coverage and documentation.
- Build new features across the stack, from the database and backend to the APIs and internal tools.
- Scale the systems from today's volume to much larger workloads, with predictable performance and cost.
- Build RL environments that agents act in, and the infrastructure that runs them at scale: sandboxes, environment images, rollouts and reward computation.
- Keep everything reliable and secure: monitoring, alerting, incident response, access control, safe handling of untrusted code and protection of client data.
- Support the data teams by building tooling that task authors, reviewers and project leads use every day, and help produce fast-turnaround samples when a project needs them.
What you'll need
- 5+ years in software, data or ML engineering, with recent work in AI/ML data, model evaluation, annotation or developer tooling.
- A track record of running and maintaining production systems, not only building them.
- Fluent English.
- Strong backend engineering in Python: clean, tested, maintainable code.
- Designing and building APIs and the data models behind them.
- Relational databases (Postgres or similar): schema design, migrations, query performance.
- Enough frontend skill (TypeScript and React) to build and maintain internal tools.
- Comfort joining a codebase someone else wrote, understanding it quickly and improving it without breaking it.
- Designing distributed systems that run many long-running jobs at once.
- Job orchestration, workflow engines and queues: retries, idempotency, failure recovery.
- Containers and container orchestration (Docker, Kubernetes or similar), and at least one major cloud provider.
- Observability: logging, metrics, tracing and alerting.
- Security fundamentals: secrets management, isolation, access control.
- Hands-on experience building RL environments for LLM agents: tool interfaces, stateful environments that can be seeded, reset and snapshotted, and reward computed from the final state or the agent's answer.
- Running agent code in isolated sandboxes at scale, with sandbox platforms such as Daytona, E2B or Modal, or with your own isolation layer (gVisor, Firecracker microVMs or similar).
- Building and caching environment images so that thousands of sandboxes can start quickly and reproducibly.
- High-throughput rollout collection: many parallel episodes, timeouts, resource limits, and full logs and trajectories for every episode.
- Feeding rollouts and rewards into RL post-training pipelines.
- Hands-on experience building or running evaluation harnesses or benchmarks for LLMs or LLM agents.
- Writing verifiers, graders or reward functions, and knowing how they fail.
- Understanding of how frontier models are evaluated: pass rates, pass@k, headroom, discrimination, reward hacking, and what makes a task hard for a frontier model.
- Working with LLM provider APIs and tracking model cost and usage.
- Experience building annotation, labeling or human-data tooling.
- Contributor and reviewer workflows, quality control and calibration.
- Turning a manual, ad-hoc process into a scaled, measured one.
- Daily, careful use of AI coding agents, reviewing their output with the same rigor as a colleague's.
- Thrives in ambiguity and ships fast; works independently and takes initiative on a small team.
- Clear written communication with both engineers and non-engineers.
Nice to have
- Contributions to open-source evaluation or RL frameworks.
- Experience building tools for a large distributed contributor workforce.
Details
- Location: Argentina, fully remote.
- Works as a core member of the platform team.
Read the full description and apply on the company’s own careers page.