Sr. Data & AI Engineer, Commercialization

Overview

Senior Data and AI Engineer on the Commercialization team, owning the reliability of a Databricks-native environment and building AI-for-Engineering tooling and observability for the team's data delivery.

What you'll do

• Build, test, and integrate the application components, tools, and utilities that keep the team's Databricks-native environment running reliably.
• Own the health of the Databricks application environment day to day, including jobs, workflows, notebooks, and Unity Catalog, identifying and resolving issues proactively.
• Design and build AI for Engineering: agents, copilots, and LLM-assisted tooling that predict and fix data quality problems and improve the team's delivery speed.
• Build the observability layer, including monitoring, alerting, and dashboards, that surfaces issues before they affect downstream consumers.
• Build accelerators and reusable frameworks (using tools such as Claude Code and GitHub Copilot) that reduce repetitive engineering work across the team.
• Develop ETL/ELT pipelines on Databricks, PySpark, SQL, and AWS (Glue, Lambda, S3, Athena) that hold up at production scale.
• Lead root cause analysis on production issues, driving fixes and process changes that prevent recurrence.
• Set the technical standards other engineers on the team will build to.
• Partner closely with the Platform Engineering team so tooling and infrastructure integrate cleanly.
• Bring CI/CD and infrastructure-as-code discipline to how the team ships.
• Mentor junior engineers on observability, automation, and applied use of AI to improve engineering delivery.

What you'll need

• A degree in Computer Science, Engineering, Data Science, Information Systems, or a related field.
• 6–8 years of experience building and owning production systems, ideally with substantial Databricks experience.
• Strong command of Python, PySpark, and SQL, sufficient to build automation and monitoring tooling, not just transformation logic.
• Real production experience running a Databricks environment, including Delta Lake internals, Unity Catalog, and Workflows.
• A track record of building AI agent or LLM tooling that has measurably improved an engineering team's delivery.
• Experience building monitoring or data-quality tooling from the ground up, rather than configuring existing dashboards.
• Experience building AI-for-Engineering tooling that has measurably improved a team's delivery.
• A strong orientation toward observability as a default practice rather than an afterthought.
• Comfort with agent frameworks (LangChain, LangGraph, CrewAI, AutoGen, or similar) and sound judgment on where they are genuinely useful.
• Working depth in AWS data services (Glue, Lambda, S3, Athena), sufficient to inform architecture decisions.
• A habit of writing maintainable code, reviewing others' code constructively, and having previously set technical standards for a team.
• Sound understanding of data governance, with the ability to build access controls in rather than add them after the fact.
• Strong root-cause-analysis discipline and the ability to communicate technical findings clearly to engineers and non-technical stakeholders.
• A track record of mentoring engineers or setting technical direction for a team.

Nice to have

• Experience with pharma commercial data (claims, sales, payer, patient, HUB) and the integration challenges typical of these datasets.
• AWS, Databricks, or related certifications.

Details

• Location: Hyderabad, Telangana, India.
• Work model: site-essential, site-by-design, field-based, or remote-by-design, as assigned based on core responsibilities (see https://careers.bms.com/ways-of-working).