Staff Data Engineer

Overview

Senior individual contributor role shaping and delivering data and AI solutions, providing technical leadership across data products, data platforms and AI-enabled solutions, from high-level architecture through to low-level implementation design. Hands-on engineering leadership without line management responsibility.

What you'll do

• Translate high-level architectural direction into detailed, buildable solution designs, including table schemas, transformation logic, interface contracts, retry and error-handling strategies, SLA thresholds and observability requirements.
• Produce HLD artefacts communicating solution intent and platform choices, and LLD artefacts that delivery engineers can implement without ambiguity.
• Set and reinforce standards for software design, code quality, testing, CI/CD, observability and documentation, applying patterns such as event-driven architecture, microservices, pipeline orchestration, RAG and layered data architecture.
• Design, build and maintain reliable data pipelines, data products and integration layers that move, process and serve data across the enterprise at scale.
• Integrate AI and machine learning capabilities into production-grade applications, including LLM-based solutions, retrieval-augmented generation, AI agents and unstructured data processing.
• Determine when to build custom capability, configure or extend existing platforms, adopt SaaS solutions, or use low-code/no-code approaches, and identify opportunities for reuse and shared services.
• Work with architecture, cyber, data governance and platform teams to ensure solutions meet enterprise standards for security, resilience, compliance, performance and operational readiness.
• Provide technical direction and engineering leadership across internal teams, delivery partners and third-party suppliers.
• Design for supportability, observability, reliability and cost effectiveness, and define service reliability expectations, SLA requirements and site-reliability engineering practices.
• Translate ambiguous problems into practical technology options, solution designs and working prototypes, and validate ideas rapidly with users.
• Mentor engineers across the team and contribute to data engineering learning and development.

What you'll need

• Bachelor's degree in Computer Science or a related field.
• 8+ years of hands-on experience designing, building, productionising and maintaining reliable, scalable data infrastructure and data products in complex environments.
• Demonstrable experience producing HLD and LLD artefacts for data or technology solutions, including solution architecture diagrams, data-flow designs, component specifications, API contracts, data models and operational runbooks.
• Ability to take architectural decisions and decompose them into engineering specifications that developers can build from without further clarification.
• Strong command of software engineering best practices, including technical design review, unit testing, CI/CD pipeline development, monitoring and alerting, code review and documentation.
• Development experience in one or more object-oriented programming languages, with Python preferred. Scala, Java or C# are also applicable.
• Expert SQL knowledge and experience working with large-scale distributed data systems.
• Experience integrating AI and machine learning capabilities into production systems, including LLMs through APIs, RAG patterns, vector search, embeddings or AI agents.
• Deep knowledge and hands-on experience across the full data lifecycle.
• Strong stakeholder management skills and the ability to lead through technical influence across internal teams and external partners.
• A continuous learning and improvement mindset.

Nice to have

• Experience with AI/ML frameworks such as LangChain, LlamaIndex, Hugging Face, Weaviate or similar ecosystems.
• Familiarity with agentic frameworks such as CrewAI, AutoGen or similar LLM orchestration toolkits.
• Experience with big data technologies such as Spark, Hadoop or Hive.
• Familiarity with cloud data platforms such as AWS, Azure or GCP, and infrastructure-as-code practices.
• Experience with data governance tools such as Alation or Collibra, and enterprise data management standards.
• No prior experience in the energy industry is required.

Details

• Location: Pune, India.
• Hybrid office/remote working.
• Up to 10% travel.
• Eligible for relocation within country.
• Advertised as full time; flexible working arrangements may be considered.