Kafka Senior Engineer - Officer

Overview

Senior Middleware Platform Engineer role leading the design, implementation and support of enterprise Kafka messaging infrastructure, including the first Kafka cluster on a public cloud.

What you'll do

• Design, engineer, and evangelize Kafka messaging across the enterprise
• Lead the design and engineering of the first Kafka cluster on a public cloud (Azure, AWS, other) using Ansible, Terraform, Python
• Lead and assist in providing direction for Junior Engineers
• Work with the Enterprise Architecture team to identify and implement use cases best suited for Kafka
• Write code to automate tasks such as request processing, product installations, health checks, proactive monitoring, alerting, reporting, and predictive analytics
• Write Kafka producer and consumer code within consumer groups to test Kafka client APIs, develop Kafka application best practices, and safely onboard Kafka applications across programming languages and platforms
• Configure and test pre-built Kafka Connectors for integrating Kafka with other systems
• Design, code, and test new Kafka Connectors where needed for integrating Kafka with other systems
• Perform optimization and tuning of Kafka
• Design monitoring solutions for the platform and baseline statistics reporting
• Create best practices, standards, integration patterns and on-boarding process automation
• Work with application development teams to understand usage patterns and provide solutions
• Collaborate with other infrastructure teams to achieve high resiliency and scalability of Kafka messaging platforms
• Provide technical expertise in scoping end-to-end business requirements and reflecting them in implementation for different Kafka connectors
• Conduct technical reviews of integration implementations for all SDLC phases to assess the quality of integration solutions built on Kafka
• Provide highly resilient Kafka messaging systems using the latest high availability (HA) and disaster recovery (DR) technology solutions
• Assist and act as final level of support for production services supported by the team
• Create new docs and SOPs for the support team and perform cross training of support team members

What you'll need

• Love of writing code and programming
• Solid coding ability to write robust, fault tolerant, well tested code; any scripting/programming language is fine with willingness to learn Python, Ansible, Terraform, Puppet, Shell
• Experience with a source code control system
• Experience working on Red Hat Linux
• Deep level of understanding of Kafka messaging or another messaging technology
• Experience finding root cause for problems with Kafka or applications using Kafka (or another messaging technology)
• Experience tuning Kafka systems (or other messaging systems)
• Experience supporting application architectures SEDA, Kappa, and Lambda or similar
• Programming and automation experience
• L4 ability to drive a project and collaborate within the team and with other application and infrastructure teams
• L4 ability to troubleshoot complex issues
• Hands on keyboard engineering position

Nice to have

• 8+ years of experience (preferred but not required depending on candidate)
• Computer science degree (preferred but not required depending on candidate)
• Experience with high availability (HA) and disaster recovery (DR) solutions for middleware or other platforms

Details

• Location: Bangalore or Hyderabad (candidates not willing to relocate need not apply)
• No hybrid working; fully working from office
• Schedule: APAC or EMEA time
• Occasional weekend work may be required for prod deployments/changes