The client is one of the EU’s largest debt management companies on a mission to help our customers take control of their debt. The client was founded following the merger of the UK, Nordic, and German market leaders.
The client was struggling with disparate data sources, which led to multiple versions of truth across business groups, reporting teams, and data science teams. Consequently, reports had to be generated manually, and accuracy validations required significant time and effort.
The existing on-premise applications were not designed to handle the size and complexity of the data. Previously, queries took weeks to complete and required multiple manual interventions due to errors. As a result, model and report deployment pipelines were delayed for years.
Additionally, following acquisitions and organic growth, there were introduced new source systems and data sets that could not be integrated into the existing data analytics landscape.
Following a comprehensive analysis of the client’s environment and necessities, we have designed and built a scalable Data Lakehouse architecture using Azure Data Lake and Databricks Delta Lake to support complex data science products, such as feature stores, sample selectors, and reporting data sets.
Brillio performed development and deployment of end-to-end models using Databricks, MLOps, DevOps, and leveraged AKS to deploy scoring models, along with Synapse to store batch scores.
Following our implementation, the client managed to achieve:
Data Lakehouse Architecture – Ensure ACID transaction on big data scale and processing, ensuring cost efficiency and performance.
