Data Engineer

Overview

Design, build and maintain the data architecture, pipelines and integrations that turn raw operational data into reliable, analysis-ready information for reporting, BI and decision support on a DHS software development program.

What you'll do

• Design logical and physical data models suited to analytical workloads.
• Develop data structures that integrate information from multiple applications, databases, files, APIs, logs and external sources.
• Evaluate existing data architectures and recommend improvements to reliability, scalability, performance, maintainability and accessibility.
• Support the design and implementation of data warehouses, data marts, data lakes, lakehouses and other analytical repositories.
• Design, develop, test, deploy and maintain ETL/ELT processes and automated data pipelines.
• Extract and integrate data from relational databases, applications, APIs, flat files, logs and other structured and semi-structured sources.
• Develop repeatable processes that transform raw transactional data into standardized, analysis-ready datasets.
• Implement scheduling, orchestration, monitoring, error handling and recovery for data pipelines.
• Develop interfaces and data feeds supporting reporting, dashboards, analytics platforms and external systems.
• Clean, normalize, deduplicate, validate and transform data.
• Investigate data discrepancies and work with application developers, DBAs, analysts and business users to determine root causes.
• Implement controls that prevent data-quality problems from propagating into downstream reports and analytical products.
• Develop curated datasets and data models optimized for BI, reporting, visualization and analytical workloads.
• Work with BI analysts and business stakeholders to translate business questions into data requirements.
• Support common definitions and calculation methodologies for enterprise metrics and KPIs.
• Support ad hoc analytical requests when new data sources or transformations are required.
• Assist analysts in understanding data lineage, relationships, limitations and appropriate use.
• Document data sources, transformations, interfaces, business rules, dependencies, lineage and definitions.
• Maintain technical documentation for pipelines, models, interfaces and analytical repositories.
• Support development and maintenance of data dictionaries and metadata repositories.
• Develop and optimize complex SQL queries, stored procedures, views and data transformations.
• Analyze and improve performance of analytical queries and data-processing workloads.
• Work with DBAs to optimize database structures supporting analytical requirements.
• Recommend indexing, partitioning, aggregation, caching or other techniques where appropriate.
• Ensure analytical workloads do not adversely affect production transactional systems.
• Implement data-engineering solutions consistent with organizational cybersecurity and privacy requirements.
• Apply access controls to sensitive data and support classification, encryption, auditing, logging, retention and protection requirements.
• Work with cybersecurity personnel to ensure data movement and storage comply with applicable security requirements.
• Support protection of sensitive information, including PII and other controlled or restricted data.
• Develop maintainable data-engineering code using established software development practices.
• Maintain pipeline code and configuration under version control.
• Participate in peer reviews, testing, configuration management and change-control processes.
• Develop automated tests for data transformations and pipelines.
• Support CI/CD and automated deployment practices for data-engineering components.
• Troubleshoot production data-processing failures and perform root-cause analysis.

What you'll need

• U.S. Citizen.
• Ability to receive a Public Trust clearance.
• Verifiable accredited education.
• Bachelor's degree in Computer Science, Data Science, Information Systems, Software Engineering, Computer Engineering, Mathematics, Statistics, or a related technical discipline.
• 3–5+ years of professional experience in data engineering, database development, software engineering, BI engineering or a related technical discipline.
• Demonstrated experience designing and developing ETL/ELT processes and data pipelines.
• Strong experience with relational databases and SQL.
• Experience integrating data from multiple heterogeneous sources.
• Experience developing analytical data models, data warehouses or similar repositories.
• Experience troubleshooting data-quality and data-integration issues.
• Strong proficiency in advanced SQL and relational database concepts.
• Strong proficiency in Oracle, SQL Server, PostgreSQL or comparable RDBMS technologies.
• Strong proficiency in Python or another data-processing/programming language.
• Strong proficiency in ETL/ELT development.
• Strong proficiency in data modeling and dimensional modeling.
• Strong proficiency in data warehouse architecture.
• Strong proficiency in REST APIs and system integration.
• Strong proficiency in data validation and quality management.
• Strong proficiency in Git or comparable version-control systems.
• Strong proficiency in automated testing and CI/CD concepts.
• Strong proficiency in structured and semi-structured data processing.
• Strong analytical and problem-solving skills.
• Ability to understand complex relationships among applications, databases, interfaces and business processes.
• Ability to translate business information requirements into technical data solutions.
• Understanding of transactional versus analytical data architectures.
• Understanding of data lineage, governance, quality, security and lifecycle management.
• Ability to communicate technical concepts to technical and nontechnical audiences.
• Ability to distinguish between data, application, business-process and analytical problems.
• Strong documentation and configuration-management discipline.

Nice to have

• Experience with technologies such as Azure, AWS, Snowflake, Databricks, Spark, Airflow, dbt, Kafka, Power BI, Tableau or similar platforms.

Details

• Location: 6314 Remote/Teleworker US.
• Pay range: $131,300.00 – $237,350.00.
• Interview requires camera to be kept on, and the interviewer will take your picture for identification if an offer is extended.