What the company needed was not a patch. It needed a modern, unified data platform, one capable of handling growing data volumes, enforcing governance at scale, and delivering analytics-ready outputs without requiring constant manual intervention from engineering teams. The gap between where the infrastructure was and where the business needed it to be was significant, and closing it required both the right technology and a partner with the expertise to implement it without disrupting ongoing operations.
Solution
Our team at Brillio demonstrated exceptional capabilities in metadata-driven ETL, Microsoft Fabric, and data governance frameworks. We proposed a platform architecture centred on Microsoft Fabric with OneLake as the central data repository. The design used three distinct layers to move data from raw ingestion through to analytics-ready outputs.
The Bronze Layer handled raw ingestion from Parquet files, preserving full data fidelity and auditability so that nothing was lost in transit. The Silver Layer applied deduplication, column standardisation, and Slowly Changing Dimension (SCD2) logic, producing clean and consistent datasets that downstream consumers could trust. The Gold Layer then introduced business logic and analytics use cases, turning that cleaned data into enriched, consumable formats for reporting and decision-making.
A metadata-driven ETL framework was central to the approach. By externalising transformation logic, quality rules, and source-to-target mappings into configuration files, we made it possible to onboard new datasets and adjust pipelines dynamically, without requiring hard-coded changes to the underlying code. This significantly reduced manual intervention and cut long-term maintenance overhead.
Microsoft Purview was integrated early, embedding governance, data lineage tracking, and quality validation into the platform’s architecture from the ground up, rather than bolting them on later.
Delivery followed a structured, collaborative model: discovery workshops, joint architecture design sessions, daily checkpoints, knowledge transfer sessions, and milestone reviews. Microsoft played an active role throughout, providing Fabric and Purview environment access and supporting key setup activities. Platform ownership was handed over gradually, ensuring the client’s teams were fully equipped to operate and extend the solution independently.
