Lead Product Engineer – (AIOps for Observability)

Overview

Lead a team to design and build AI-powered observability and AIOps solutions, including agentic AI for anomaly detection, root cause analysis and automated remediation, across hybrid and multi-cloud environments.

What you'll do

• Design and build agentic AI solutions for observability, including autonomous agents for anomaly detection, root cause analysis and automated remediation.
• Embed AI/ML capabilities into observability platforms to enable predictive insights, anomaly detection and intelligent alerting.
• Develop AIOps frameworks that reduce noise, automate workflows and improve incident response times.
• Leverage LLMs and AI orchestration frameworks to create self-service diagnostics and operational assistants.
• Architect and implement end-to-end observability solutions across logs, metrics, traces and events.
• Enhance observability platforms (Datadog, Dynatrace, New Relic, AppDynamics, OTEL) with custom integrations, automation and AI enhancements.
• Build real-time insights and health analytics to ensure system reliability, scalability and performance.
• Lead development of scalable, reusable platform components and APIs for observability and automation.
• Build developer-first tools and frameworks that simplify instrumentation, monitoring and diagnostics.
• Apply modern software engineering practices including TDD, CI/CD, microservices and cloud-native design.
• Develop full-stack solutions (backend services, dashboards, automation tooling) using Java/Spring Boot, Python, Node.js and React.
• Drive automation-first strategies to eliminate manual operations and improve efficiency.
• Build self-healing systems that automatically detect and remediate issues.
• Lead initiatives in infrastructure-as-code, Observability-as-code and automated deployment pipelines.
• Engineer highly resilient systems by focusing on performance, scalability and fault tolerance.
• Define and evolve enterprise observability architecture, incorporating AI and automation as core principles.
• Collaborate with architecture, platform and security teams to align solutions with enterprise standards.
• Champion standardization, reusable patterns and platform scalability across the organization.
• Act as a technical leader and mentor, guiding teams in AI, observability and engineering best practices.
• Partner with cross-functional teams to deliver integrated, high-impact platform solutions.
• Communicate technical concepts and strategies effectively to both technical and executive audiences.
• Foster a culture of innovation, automation and continuous improvement.

What you'll need

• 5+ years of engineering experience building software, platforms or automation solutions, with a strong emphasis on observability and distributed systems.
• 4+ years of hands-on experience with observability platforms (Datadog, Dynatrace, OTEL).
• 3+ years of software development experience using Java (Spring Boot), Python, Node.js or React.
• 2+ years of hands-on experience building agentic AI systems (autonomous agents, AI workflows or AI-native applications).
• Proven ability to design and implement AI-driven automation and AIOps solutions.
• Experience integrating LLMs, orchestration frameworks or AI pipelines into production systems.
• Strong experience with Kubernetes and cloud-native architectures.
• Hands-on experience in hybrid environments (on-prem and cloud) across Linux and Windows.
• Expertise in API development, event-driven systems and microservices architecture.
• Familiarity with CI/CD pipelines, infrastructure-as-code and DevOps practices.
• Strong problem-solving skills with a focus on automation and innovation.
• Excellent communication skills to articulate complex technical solutions.
• Passion for emerging technologies, particularly AI in observability and operations.

Details

• Location: USA, Illinois (Remote).
• This job does not have supervisory duties.
• Allstate generally does not sponsor individuals for employment-based visas for this position.
• Compensation is 100,000.00 - 170,500.00 annually, based on experience and qualifications.
• Successful candidates will be required to submit to a background investigation.