G

Senior Software Security Engineer - AI Benchmark Development

Gramian
NewPosted today

LOCATION

Colombia · Remote

EXPERIENCE

5+ Years

TYPE

Contract

SKILLS REQUIRED

Vulnerability AnalysisSecure CodingRoot Cause AnalysisDebuggingOWASP top 10CWE Classification

Job description

Overview

Develop Harbor-format audit-and-patch benchmark tasks from real-world software vulnerabilities to evaluate whether AI coding models can identify, reproduce and correctly fix security issues.

What you'll do

  • Source and investigate real-world vulnerabilities from CVEs, security advisories, bug reports, and open-source repositories.
  • Identify and classify vulnerabilities using the most specific applicable CWE and map them to relevant OWASP Top 10 categories where appropriate.
  • Reproduce security issues and develop reliable proof-of-vulnerability tests that demonstrate the underlying weakness.
  • Build Harbor-format audit-and-patch tasks across multiple programming languages and software ecosystems.
  • Write realistic task instructions that describe the security issue without revealing vulnerable code, exploit details, or the expected remediation.
  • Create reproducible Docker-based environments with pinned dependencies, clear setup instructions, and reliable execution procedures.
  • Design and harden automated verifiers to prevent gaming, test manipulation, hardcoded solutions, and overfitting.
  • Review and score AI-generated patches and technical reasoning using structured security and engineering rubrics.
  • Identify tasks that can be solved through superficial pattern matching or copying known upstream patches without understanding the vulnerability.
  • Document vulnerability context, reproduction steps, expected behavior, patch requirements, test methodology, and review decisions.

What you'll need

  • 5+ years of professional software engineering experience.
  • Expert-level proficiency in at least one of the following languages: Python, JavaScript / TypeScript, Java, C / C++, C#, Go, Rust, Ruby, PHP, Kotlin, Swift.
  • Strong understanding of software security principles and common vulnerability classes.
  • Practical knowledge of CWE, CVE, and OWASP Top 10.
  • Ability to independently reproduce vulnerabilities and develop proof-of-vulnerability tests.
  • Hands-on experience with security testing and analysis tools, including sanitizers, fuzzers, and static-analysis tools such as CodeQL or Semgrep.
  • Strong proficiency with Git, Linux, Bash, and Docker.
  • Ability to create automated tests and reproducible technical evaluation environments.
  • Excellent technical writing skills, with the ability to explain complex security issues and remediation requirements clearly.
  • Ability to work independently in a fully remote, research-oriented environment.

Details

  • Contractor assignment, 3 months.
  • Expected to start next week.
  • Commitment: 40 hours per week.
  • Fully remote work arrangement.

Read the full description and apply on the company’s own careers page.

Stay safe

Hiring on Abekus is free for applicants

We never charge a fee, and employers are prohibited from doing so. If a recruiter asks for payment, please report them right away.

Senior Software Security Engineer - AI Benchmark Development