Overview
As a Data Engineering Consultant specializing in GCP Architecture, you will design, build, and optimize cloud-native data solutions on Google Cloud Platform.
What you'll do
- Design and implement end-to-end data architectures on GCP leveraging BigQuery, Dataflow, Pub/Sub, Vertex AI Pipelines, and Gemini API.
- Build scalable, serverless streaming and batch data pipelines using Apache Beam and Cloud Dataflow.
- Integrate Gemini API for generative AI use cases within data products and analytical workflows.
- Orchestrate ML training and inference pipelines on Vertex AI, ensuring reliability and performance at scale.
- Configure real-time event-driven architectures using Pub/Sub for data ingestion and messaging.
- Collaborate with data scientists, analysts, and client stakeholders to define data platform requirements.
- Conduct performance tuning, cost optimization, and governance reviews for GCP environments.
- Stay current with GCP product releases and AI/ML advancements, recommending adoption where impactful.
What you'll need
- Proficiency in GCP services: BigQuery, Dataflow, Pub/Sub, Vertex AI Pipelines, and Gemini API integration.
- Hands-on experience in PySpark, SQL, and Python for pipeline development.
- Hands-on experience designing and deploying streaming and batch data pipelines on GCP.
- Strong SQL and Python skills for data transformation and pipeline development.
- Experience with Vertex AI Pipelines for orchestrating end-to-end ML workflows.
- Knowledge of IAM, VPC, and GCP security best practices for data platforms.
- Familiarity with CI/CD tooling and infrastructure-as-code (Terraform or Deployment Manager).
- Demonstrable experience delivering GCP-based solutions in consulting or enterprise environments.
Nice to have
- Exposure to dbt or similar transformation frameworks.
Details
- Location: Bengaluru (PAN India offices).
Read the full description and apply on the company’s own careers page.