Associate Data Engineer

Overview

Build and enhance the data lake that processes billions of events daily, developing cost-effective cloud data solutions for identity security and cybersecurity research, working alongside experienced engineers.

What you'll do

• Contribute to data modelling efforts that support identity security use cases and analytical requirements.
• Develop and maintain data pipelines that bring together complex information from disparate systems to support security analysis in the datalake.
• Optimize data workloads at a software level by improving processing efficiency.
• Work with senior engineers to identify areas for process improvement and help implement solutions.
• Use monitoring and observability best practices to ensure optimal pipeline performance.
• Assist with ML Operations to ensure optimal model efficiency.
• Leverage CI/CD best practices to effectively develop and release source code.
• Participate in code reviews and collaborate with team members to improve engineering practices.

What you'll need

• Programming and technology knowledge in cloud data processing.
• Spark experience.
• Ability to learn fast and strong desire to develop.
• Understanding of software engineering principles, data structures, and distributed data processing concepts.
• Experience working with distributed systems and data processing on object stores.
• Ability to work autonomously and collaboratively.
• Required: Data warehousing for analytics use cases.
• Required: Proficient in Python (or another object orientated language).
• Required: Distributed processing framework experience.

Nice to have

• Existing Databricks knowledge.
• Interest in cybersecurity; interest in identity security is preferred.
• Highly Preferred: Spark.
• Highly Preferred: Realtime processing experience.
• Ideal: Some experience with graph data stores.

Details

• Remote, United States.