Data Platform Engineer

Overview

Data Platform Engineer role at EarnIn to scale data infrastructure, streamline data delivery, and build an enterprise-grade Data Platform, with a focus on AI automation that replaces manual operations.

What you'll do

• Lead and focus on scaling the data infrastructure, streamlining data delivery, and designing and implementing AI automation to replace manual operations.
• Engage closely with executive, product, data, and design teams, providing high-level support for data infrastructure challenges, advising on technical data issues, and implementing self-serve solutions.
• Develop sophisticated AI-based analytical tools that integrate with the data pipeline to provide deep insights into key business metrics, including customer growth and operational efficiency.
• Architect and manage large, complex datasets to meet business needs and requirements.
• Collaborate with other engineers and provide thoughtful guidance through code, design, and architecture reviews.
• Contribute to defining technical direction, planning the roadmap, escalating issues, and synthesizing feedback to ensure team success.
• Estimate and manage team project timelines and risks.
• Participate in hiring and onboarding for new team members.
• Lead cross-team engineering initiatives.
• Constantly learn about new technologies and industry standards in data engineering.

What you'll need

• 3+ years of experience in designing, building, and maintaining data infrastructure, and the ability to lead complex projects.
• Bachelor's, Master's, or PhD degree in computer science, computer engineering, or a related technical discipline, or equivalent industry experience.
• Deep knowledge of Kafka.
• Proficiency in programming languages like Python and Scala.
• Good understanding of ML engineering.
• Strong knowledge of distributed computing frameworks, such as Apache Spark.
• Strong knowledge of cloud platforms such as AWS, Azure, or GCP.
• Experience using AI-assisted development tools (e.g., GitHub Copilot, Cursor, ChatGPT, or similar tools) as part of your software development workflow.
• Deep understanding of database design, SQL, and NoSQL databases.
• Experience in managing large datasets and optimizing database performance.
• Proficiency in Git and Terraform, and experience in deploying continuous integration and continuous deployment (CI/CD) practices.
• Experience in managing event-driven systems, particularly with Kafka in cloud environments.
• Expertise in developing and implementing data governance frameworks, policies, and procedures to ensure data quality, compliance, and effective data management practices.
• Deep understanding of data security principles, including encryption, decryption, and secure data storage and transfer protocols.

Nice to have

• Working experience with Databricks.

Details

• Location: Bengaluru, India.
• Work mode: Hybrid, with 2+ days in the office.
• Opportunities to collaborate with and travel to the Mountain View HQ and Bangkok Site.