G

Lead Software Security Engineer - AI Benchmark Delivery & Team Leadership

Gramian
NewPosted today

LOCATION

Colombia · Remote

EXPERIENCE

6+ Years

TYPE

Contract

SKILLS REQUIRED

Vulnerability ManagementTechnical LeadershipStatic AnalysisProject ManagementPenetration Testing

Job description

Overview

Lead a team of trainers developing high-quality training and evaluation tasks for frontier AI coding models, transforming real-world open-source vulnerabilities into Harbor-format audit-and-patch tasks across multiple programming languages.

What you'll do

  • Plan task deliveries by batch, assign work across pod members, and track daily throughput against phase-specific targets.
  • Monitor pod performance, identify bottlenecks, resolve delivery issues, and ensure tasks move through production stages on schedule.
  • Lead weekly client reporting, communicate delivery status, and proactively escalate risks involving tooling issues, rate limits, safety blocks, and resource constraints.
  • Collaborate with the client to clarify requirements and align on task scope, volume, complexity, and difficulty changes.
  • Conduct final reviews of completed tasks to ensure they meet required quality, security, reproducibility, and evaluation standards.
  • Train and calibrate Pod Leads and trainers on CWE classification, verifier robustness, task difficulty, technical requirements, and acceptance criteria.
  • Use throughput and quality data to forecast staffing needs across project phases.
  • Initiate hiring and onboarding waves approximately three weeks ahead of projected capacity requirements.
  • Step in to research vulnerabilities, reproduce security issues, develop proof-of-vulnerability tests, and create or review benchmark tasks during periods of high workload.
  • Maintain clear documentation of delivery plans, risks, quality findings, throughput metrics, staffing forecasts, and project decisions.

What you'll need

  • 6+ years of professional software engineering experience.
  • Expert-level proficiency in at least one of: Python, JavaScript / TypeScript, Java, C / C++, C#, Go, Rust, Ruby, PHP, Kotlin, Swift.
  • Strong understanding of software security concepts, including CWE, CVE, and OWASP Top 10.
  • Ability to reproduce software vulnerabilities and develop proof-of-vulnerability tests.
  • Hands-on experience with security tooling, including sanitizers, fuzzers, and static-analysis tools such as CodeQL or Semgrep.
  • Strong proficiency with Git, Linux, Bash, and Docker.
  • Strong technical judgment when evaluating verifier robustness, security patches, and AI-generated code.
  • Experience reviewing technical work and maintaining consistent quality standards across multiple contributors.
  • Some experience mentoring engineers, trainers, or technical contributors.
  • Excellent technical writing, documentation, and communication skills.
  • Ability to work independently in a fully remote, research-oriented environment.
  • Proven track record working on this project or equivalent software security dataset, benchmark development, or security evaluation work.

Details

  • Contractor assignment, 3 months, 40 hours per week.
  • Fully remote.
  • Minimum 4 hours of overlap with PST.
  • Expected to start next week.

Read the full description and apply on the company’s own careers page.

Stay safe

Hiring on Abekus is free for applicants

We never charge a fee, and employers are prohibited from doing so. If a recruiter asks for payment, please report them right away.

Lead Software Security Engineer - AI Benchmark Delivery & Team Leadership