Overview
Data Engineer I role on Kaseya's Data Services team, focused on building and maintaining data pipelines and Silver and Gold-layer datasets using Medallion Architecture.
What you'll do
- Develop, maintain, and enhance data pipelines using Python, SQL, Snowflake, and Apache Airflow.
- Build and maintain ETL/ELT workflows to ingest data from multiple source systems.
- Develop data extraction, transformation, and loading processes to support enterprise data requirements.
- Implement incremental data loading and data processing mechanisms.
- Write reusable and efficient Python scripts and SQL transformations.
- Ensure data pipelines are reliable, scalable, and maintainable.
- Develop Silver-layer datasets by implementing data cleansing, standardization, deduplication, validation, and transformation logic.
- Build Gold-layer datasets by applying business rules, joins, aggregations, and transformations to create curated, business-ready datasets.
- Develop and maintain Fact and Dimension tables to support reporting and analytics requirements.
- Implement data quality checks and validation rules across Silver and Gold datasets.
- Ensure data consistency, accuracy, completeness, and reliability across different data layers.
- Support the development of reusable data models and transformation frameworks.
- Optimize SQL queries and data transformations for efficient dataset processing.
- Develop and maintain Snowflake tables, views, and SQL-based transformation workflows.
- Write complex SQL queries, stored procedures, and data transformation logic.
- Work with Snowflake features such as Streams, Tasks, and Dynamic Tables.
- Perform basic query optimization and troubleshoot performance-related issues.
- Support data loading, transformation, and data validation activities.
- Follow Snowflake security and access management best practices.
- Develop Python scripts and modules for data processing, integration, and automation.
- Create and maintain Apache Airflow DAGs for data pipeline orchestration.
- Configure task dependencies, scheduling, retries, and error-handling mechanisms.
- Monitor Airflow workflows and troubleshoot pipeline failures.
- Develop reusable components to improve pipeline efficiency and maintainability.
- Utilize AWS cloud services for data storage, integration, and processing, such as Amazon S3, AWS Lambda, Amazon SQS and SNS, AWS IAM, Amazon CloudWatch, and AWS Glue (preferred).
- Support integration between AWS services and Snowflake data pipelines.
- Follow cloud security and data access best practices.
What you'll need
- Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.
- 3-7 years of experience in Data Engineering, Data Development, or a related technical role.
- Good hands-on experience with SQL, including complex queries, joins, analytical functions, and data transformations.
- Hands-on experience with Python for data processing, automation, and pipeline development.
- Working experience with Snowflake, including tables, views, and SQL transformations.
- Working experience with Snowflake, including data loading and unloading.
- Working experience with Snowflake, including data modelling and schema design.
- Working experience with Snowflake, including query optimization and performance tuning fundamentals.
- Hands-on experience with Apache Airflow, including DAG development, scheduling, task dependencies, and workflow monitoring.
- Hands-on experience with AWS cloud services (Mandatory), preferably S3, Lambda, SQS, SNS, IAM, and CloudWatch.
- Understanding of Medallion Architecture (Bronze, Silver, and Gold layers).
- Practical experience developing Silver and Gold-layer datasets, including data cleansing, standardization, business transformations, joins, and aggregations.
- Good understanding of ETL/ELT processes, data warehousing concepts, and data integration.
- Understanding of dimensional modelling concepts, including Fact and Dimension tables, Star Schema, and Snowflake Schema.
- Basic understanding of incremental data loading, CDC, and historical data handling.
- Familiarity with Git and version control practices.
- Strong analytical, problem-solving, and troubleshooting skills.
- Good communication skills and the ability to work effectively within a collaborative engineering team.
Nice to have
- Familiarity with Snowflake Streams, Tasks, Dynamic Tables, and Stored Procedures.
- Exposure to CI/CD pipelines and GitHub Actions.
- Familiarity with data governance, data security, and data quality practices.
- Exposure to data observability and pipeline monitoring frameworks.
- Experience working with large datasets and optimizing data transformation workflows.
Details
- Location: Bangalore, India.
Read the full description and apply on the company’s own careers page.