Principal Software Engineer, DevOps

Overview

Principal Software Engineer on Legion's SRE/DevOps team, building the tools, infrastructure and services for a secure, scalable, cost-effective AWS/Kubernetes-based cloud platform that keeps Legion's WFM platform running for customers.

What you'll do

• Support Legion's public cloud platform, utilizing multiple cloud services and containerization.
• Develop infrastructure automation leveraging Terraform, Chef, Jenkins, and Golang.
• Create and manage production alerts.
• Respond to alerts and conduct root-cause investigations.
• Develop automated operational runbooks.
• Support the deployment of Legion's WFM solution during and off regular office hours.
• Participate in the on-call rotation.
• Build and operate internal AI agents that reduce SRE toil, including incident triage, runbook execution, alert enrichment, and capacity and cost analysis, and own their guardrails, rollback and reliability.
• Own the platform that lets other engineers build agents safely: sandboxed execution, scoped credentials, tool/MCP integrations, cost controls, and agent observability.

What you'll need

• 12+ years of experience in SRE, DevOps, or other SaaS operations.
• 3+ years of hands-on experience managing AWS services and cloud infrastructure using Terraform.
• 3+ years of experience with containerized cloud solutions using Docker, Kubernetes, or AWS EKS.
• Familiarity with Helm charts.
• 5+ years of experience with at least one programming language such as Go, Python, Bash, or Perl.
• 5+ years of experience with one or more infrastructure automation tools (Terraform, Ansible, etc.), CI/CD pipelines (Git, Jenkins, etc.), and configuration management tools (Ansible, Chef, etc.).
• Hands-on experience with one or more Linux/Unix platforms such as RedHat/CentOS/Ubuntu/Amazon Linux.
• Hands-on experience building agentic AI systems in production, including tool integrations, evaluation, and failure handling, beyond use of AI coding assistants.
• Bachelor's or Master's degree in Computer Science/Engineering or a related field.

Nice to have

• AWS Certification.
• Track record of introducing AI into an engineering organization's SDLC with measurable results, and the judgment to name where it should not be used.
• 3+ years of experience with observability tools such as Splunk, Nagios, Elasticsearch, Kibana, CloudWatch, and Logstash, and ways to scale these systems.
• 3+ years of experience with AWS RDS or Aurora MySQL.
• Demonstrated ability to work with remote teams.

Details

• Location: Remote, United States.
• Includes participation in an on-call rotation.
• Base salary range: $220,000 - $255,000, plus stock equity.