Data Engineer (Remote, US)

Overview

Allstate's Risk Data Team is seeking a Data Engineer to build the trusted data foundation for faster decisions, proactive risk management, and improved control effectiveness.

What you'll do

• Design, build, and maintain trusted risk data products and scalable batch and streaming data pipelines using cloud-native technologies.
• Integrate data from risk platforms, control systems, vendor tools, operational applications, and external sources into governed, analytics-ready datasets.
• Develop and optimize ETL/ELT workflows that automate data ingestion, transformation, validation, reconciliation, and delivery.
• Build and manage data processing workloads within modern data lake and lakehouse environments, including Microsoft Fabric and OneLake.
• Implement data quality, monitoring, lineage, and reconciliation processes to ensure data reliability, consistency, and accuracy.
• Create curated datasets and analytical outputs that support operational reporting, leadership dashboards, trend analysis, risk identification, and decision-making.
• Optimize data architectures, schemas, and processing patterns for scalability, performance, resilience, and cost efficiency.
• Develop reusable frameworks, engineering standards, CI/CD pipelines, automated testing, and operational monitoring to improve productivity and maintainability.
• Partner with analytics, product, risk, governance, security, and compliance stakeholders to establish data definitions, metrics, quality standards, and secure data access.
• Participate in Agile delivery activities including sprint planning, backlog refinement, design reviews, and continuous improvement initiatives.

What you'll need

• 4+ years of experience as a Data Engineer or in a similar role building and supporting production-grade data pipelines and data products.
• Hands-on experience with Apache Spark for large-scale data processing and transformation.
• Strong proficiency in Python, SQL, and modern data engineering best practices.
• Experience developing ETL/ELT solutions within cloud-based data lake, lakehouse, or analytics platforms.
• Experience designing and optimizing analytical data models that support reporting, dashboards, operational insights, and advanced analytics.
• Strong understanding of data quality, validation, monitoring, reconciliation, and production support processes.
• Experience with CI/CD pipelines, version control, automated testing, and infrastructure-as-code concepts.
• Demonstrated ability to troubleshoot complex data quality, performance, and integration issues across multiple data sources.
• Strong verbal and written communication skills with the ability to explain technical concepts to both technical and non-technical audiences.
• Experience with risk, controls, compliance, audit, governance, or regulatory data domains.
• Experience building real-time, near real-time, or event-driven data processing solutions.
• Familiarity with data governance, metadata management, data lineage, access controls, and enterprise data quality frameworks.
• Experience with orchestration and workflow management tools for data pipelines.
• Experience supporting operational reporting, executive dashboards, trend analysis, advanced analytics, or machine learning initiatives.

Nice to have

• Experience with Microsoft Fabric, OneLake, or similar modern analytics platforms.
• Interest in building governed, scalable datasets that support AI and data-driven products.

Details

• Location: USA - IL (Remote).
• Compensation: 100,000.00 - 170,500.00 annually, based on experience and qualifications.
• The offered candidate will be required to submit to a background investigation.
• Allstate generally does not sponsor individuals for employment-based visas for this position.
• Reliable internet is required, with minimum speeds of 50 MB download and 5 MB upload, and a dedicated, private workspace is required when working from home.
• Candidates should join interviews with camera on and have valid photo identification available at the start of the interview.