Google Cloud Migration Cost Savings for a Global CPG Leader
A global FMCG / CPG company needed one platform to track customer interactions across web, mobile app, CIAM and loyalty touchpoints. DS Stream migrated its data to Google Cloud Platform and built a Consumer Data Platform on Dataproc, BigQuery and Pub/Sub. The outcome: Google Cloud migration cost savings and a near real-time view of each customer.
Results at a glance
- The Consumer Data Platform on Google Cloud Platform is the single source of accurate data for all BI and ML models.
- The platform ingests hundreds of millions of customer events every day.
- Marketing technologists moved away from isolated data storage to a comprehensive, real-time understanding of each customer.
- The migration to Google Cloud Platform resulted in cost savings for the client.
- The platform became a foundation for customer insights and a worldwide data-driven strategy for the whole organization.
What did the global CPG company need from its data platform?
The company needed a Data Platform that monitors customer interactions across many touchpoints and gives real-time insight into customer behavior and product usage. It had to serve both Machine Learning and Business Intelligence teams, and collect data from several very different sources in one place.
Data sources the platform had to gather
- Web clicks
- Mobile app clickstream
- CIAM events
- Interactions from the Loyalty Management System
- Events from a SaaS CDP
How did DS Stream move the data to Google Cloud and load it?
DS Stream moved parquet files from another cloud storage provider into Google Cloud Storage buckets with the STS service. Each data flow then got its own ingestion path into BigQuery: batch, event and real-time. The team also built orchestration for the ETL processes and the downstream SQL processing.
Ingestion paths on Google Cloud Platform
- Migrated files: parquet files transferred to Google Cloud Storage buckets with the STS service.
- Flat files: ingested and transformed with pySpark in Dataproc, then loaded into BigQuery.
- SaaS CDP events: loaded with batch Dataflow jobs.
- Real-time sources: ingested through Pub/Sub, with BigQuery as the sink.
- Orchestration: ETL processes and downstream SQL processing run as orchestrated workflows.
Read more about our approach to cloud migration and data pipeline architecture.
What changed for the business after the migration?
The company replaced a fragmented, siloed data approach with a near real-time understanding of each individual customer. The Consumer Data Platform now feeds every BI and ML model with accurate data, and it handles hundreds of millions of customer events per day on Google Cloud Platform.
Why it worked
- The solution relied on the scalability, stability and security features of Google Cloud Platform.
- One trusted data source removed the need to reconcile isolated stores.
- The platform is now a core pillar for customer insights and for the organization's global data-driven strategy.
Before and after the Google Cloud migration
| Aspect | Before | After |
|---|---|---|
| Data location | Parquet files with another cloud storage provider | Google Cloud Storage buckets, loaded into BigQuery |
| Data storage approach | Isolated data storage | One Consumer Data Platform on Google Cloud Platform |
| Customer view | Fragmented data | Comprehensive, near real-time view of each customer |
| Data for BI and ML | Spread across separate sources | Single source of accurate data for all BI and ML models |
Questions about this migration
Which Google Cloud services did DS Stream use in this project?
The platform uses Google Cloud Storage buckets and the STS service for file migration, Dataproc with pySpark for flat-file processing, Dataflow batch jobs for SaaS CDP events, Pub/Sub for real-time sources and BigQuery as the central data store and sink for all flows.
Does this case study state the size of the cost savings?
No. The case study reports that the migration to Google Cloud Platform resulted in cost savings for the client, but it does not publish a figure or percentage. The measurable result it does give is scale: the platform ingests hundreds of millions of customer events every day.
Was the client's data already in the cloud before the migration?
Yes. The parquet files sat with another cloud storage provider. DS Stream moved them to Google Cloud Storage buckets using the STS service, then processed and loaded the data into BigQuery, so this was a cloud-to-cloud migration combined with a new consumer data platform.
Related case studies and services
- Data Platform with a Near Real-Time Customer View (Global CPG Leader)Another CPG data platform project focused on a holistic, near real-time view of each customer.
- Cloudera to GCP Migration for a Global CPG LeaderA migration of data pipelines and database structures to Google Cloud services for a CPG company.
- Data Migration to Google Cloud BigQuery for an E-commerce LeaderA BigQuery migration in e-commerce, useful if your data sits outside CPG.
- Cloud MigrationHow DS Stream plans and delivers moves to the cloud.
- Data Engineering servicesThe service line behind this project: pipelines, ingestion and data platforms.
- All case studiesBrowse DS Stream projects across FMCG, retail, e-commerce and more.
Planning a migration to Google Cloud?
Tell us about your data sources and goals. Dominik Radwański, Service Delivery Partner, and our team reply within 4 hours on working days (Mon-Fri, 9am-5pm CET).
Contact DS Stream