Centralized GCP MLOps Platform for Cost Efficiency and Streamlined Development
A global FMCG / CPG company ran multiple production ML use cases across various cloud platforms. This made resource use and maintenance inefficient. DS Stream built a centralized GCP MLOps platform with automated CI/CD, auto-ticketing, autoscaling and drift-triggered retraining. The client gained cost-efficient, agile ML operations, improved scalability and reduced downtime.
Key results
- The client's ML use cases moved from a fragmented, multi-cloud setup onto one centralized platform on Google Cloud Platform (GCP).
- Resource inefficiencies dropped significantly, supported by resource autoscaling and consumption logging.
- The client achieved improved scalability, reduced downtime and greater operational agility.
- The ML development lifecycle is now standardized, with automated CI/CD that follows organizational standards and best practices.
- Failed ML pipelines now trigger tickets and emails for the operations team automatically.
Why did the client need a centralized MLOps platform?
The client had to develop, deploy and maintain multiple production-ready ML use cases spread across various cloud platforms. This fragmented approach wasted resources and increased maintenance effort. The company wanted one scalable platform for all ML operations, plus faster and simpler development and deployment through effective CI/CD practices.
What did DS Stream build on Google Cloud?
DS Stream designed and implemented a flexible, scalable MLOps platform on Google Cloud Platform, tailored to the client's operational needs. It brings the client's diverse ML use cases into one place and automates the work that used to be spread across teams and clouds.
Key components
- Automated CI/CD: build and deployment processes that follow the client's organizational standards and best practices.
- Auto-ticketing for operations: when an ML pipeline fails, the platform sends emails and opens tickets for the operations team.
- Resource autoscaling and consumption logging: resources scale with demand, and usage is logged to keep consumption efficient.
- Model lifecycle automation: model versioning, monitoring and retraining pipelines. Retraining starts automatically when data drift or concept drift is detected.
How did the team deliver and standardize the platform?
The team ran one comprehensive project to centralize the client's ML use cases on GCP. It built on GCP infrastructure and services and used Docker, Kubernetes and CI/CD to standardize the ML development lifecycle. The result is a single platform where models are developed, deployed and maintained the same way.
Which technologies power the platform?
The case study lists this technology stack for the centralized MLOps platform:
- GCP (Google Cloud Platform)
- Kubeflow
- BigQuery
- Kubernetes
- Python
- Docker
- CI/CD with GitHub Actions
The service delivered was MLOps. For more on this service, see our MLOps services.
What changed for the client's ML operations?
| Area | Before | After |
|---|---|---|
| Where ML use cases run | Spread across various cloud platforms | One centralized platform on GCP |
| Resource use | Inefficient resource utilization | Autoscaling and consumption logging; inefficiencies significantly reduced |
| Maintenance | High maintenance effort from a fragmented setup | Streamlined development and operational workflows |
| Development and deployment | Needed streamlining through effective CI/CD | Automated CI/CD aligned with organizational standards |
| Operations | Fragmented operations | Improved scalability, reduced downtime and greater agility |
Questions about this MLOps project
What happens when an ML pipeline fails on the platform?
The platform includes an auto-ticketing system for the operations team. When an ML pipeline fails, it automatically sends emails and creates tickets. The operations team learns about failures without manual checks and can respond from a single place instead of tracking issues across several cloud environments.
Does the platform retrain models automatically?
Yes. The platform includes model versioning, monitoring and retraining pipelines. Retraining is triggered automatically in response to data drift or concept drift, so models are refreshed when the data or the patterns they learned from change, without waiting for a manual decision.
Which client was this project for?
The client is a global FMCG / CPG company. The case study does not name it. The project centralized the company's machine learning use cases, which had been running across various cloud platforms, onto Google Cloud Platform.
Related MLOps work and services
- MLOps servicesThe service behind this project: platforms and practices for running ML in production.
- MLOps ImplementationHow DS Stream sets up MLOps platforms like the one in this case study.
- Standardizing FMCG Operations with MLOps Best Practices Across CloudsAnother FMCG case study on bringing consistent MLOps practices to several cloud platforms.
- MLOps Platform for Scaling Deep Learning Training and InferencingA related case study on scaling model training and inference on an MLOps platform.
- FMCG industryData and AI work for FMCG and CPG companies, the sector of this client.
- All case studiesBrowse more DS Stream projects across industries and cloud platforms.
Running ML use cases across several clouds?
Tell us about your ML setup. We'll get back to you within 4 hours on working days (Mon – Fri, 9am – 5pm CET).
Contact DS Stream