A single source of truth and self-service reporting
The client needed high data processing, increased flexibility, and wanted to become a self-serve analytics-driven organization capable of leveraging industrial-grade analytics models.
The absence of a single source of truth and data consolidation
Inconsistent data quality across business lines and limited data sets
Lack of self-service features in contrast to the current report-oriented scenario
Brillio proposed an AWS-based cloud data platform. For the implementation, we leveraged the Brillio catalog accelerator for one-time metadata creation, along with pyspark for data merge. We also helped with self-service data preparation capability using a template-based approach, and enabled metrics computed using Athena for automated feed to consumption layer – Tableau, data feeds, and APIs.
Brillio also implemented an automated test framework using APTA – Brillio testing accelerator to comprehensively manage data quality for effective build and maintenance, and future data science use cases using multiple domains of data addressed – business, product data, consumer data, lead data, marketing data, etc.
Built AWS cloud data lake to act as a single source of truth for heterogeneous data, batch data, and streaming data
Handled historical data migration to AWS S3 RAW zone, and leveraged AWS redshift for DWH purposes, streaming data capabilities with Kinesis, in-memory processing using Spark, etc.
Leveraged AWS Glue and python shell to perform ETL, data processing, derive data set and Athena for metrics computation
Built templates in python for data transformation to get the processed transactional data (PD), business data (BD), and access data (AD)
Report rationalization of reports using Brillio CREDTM
Standardized metrics consumption across org through APIs and BI tools with Tableau server, as well as integration with self-service reporting via Redshift
Centralized development & distribution of reports
Reporting consolidation using Brillio CREDTM framework
Athena and Glue to build business KPIs for streamlined insight generation
Brillio Automated testing and DQ framework for data consistency and quality
Business Benefits & Impact
Following the implementation, the client immediately started seeing a significant improvement, from a 4x ROI increase – with a profit of $28 mil, to much more streamlined and optimized data operations.
ROI of 4x times the investment (Approx profit of ~$28M)