2025-12-31

Yet another year in The Data Warehouse - 2025

The Data Warehouse is still live and kicking. 25th year anniversary is coming 2026. From a very modest start 2001 it has a steady growth until about 2018 both in terms of users (+300) and workflows (+3000/day). Workflows extracting data from ERP systems massaging data, creating reports, apps, mails and acting as a data hub for external apps and GUIs. Now users and workflows are decreasing but not as fast as we wish (I'm a bit tired of  the DW, but I will maintain it until it is decommissioned, no date sat yet. 
 This year we upgraded all servers to Ubuntu 2022.04 for security reasons and I'm planning to upgrade MySQL for the same reason. I will also make an attempt to upgrade PHP to vs 8.4 if there is not too much work. These upgrades has been planned for a very long time, but there is always things coming up.  Data Warehouses are very dependent of their surroundings and thus vulnerable to disturbances in the IT landscape. Plus, most apps are never done; they evolve until they are dead. This includes the dying DW.  
 I am responsible for a stupid workflow updating currencies. At the time the currency workflow was created, it was almost decent, a manual process of extracting currencies from an Excel and downloading some extra currencies from SAP. This has to be done on the last working day of the month. Now in the last hours of 2025 I am together with an Indian colleague waiting for the SAP ERP system to update some currencies before we can update the DW currencies. this will happen around 2 pm CET, which means new year parties already started in India.  
 The Data Warehouse is written mainly in spare time between 2001 and 2006 with no proper maintenance since 2012. And it shows, in 2012 the DW was in a league of its own. Today there are a number of options that are as good or better. Two friends of mine (Camilla Hillero and Andreas Roine-Larsson)  have created successful data -warehouse/-lake/-lake house or whatever they are called, using Snowflake and Clickhouse, they are equally pleased of their own choices. 

 At last a happy new year and my best whishes for 2026😀 

No comments:

Post a Comment