Another Year Has Passed

It's been a while since I wrote a post now, this year is not so much to write home to mum about. Not much €work, not much work at all really. Since the Covid struck business have been bad. But more work than Yesteryear, actually a little more work than I want. I'm finishing an upgrade of my Data Warehouse to PHP 7 (and 8.1 soon). The big problem was the SAP connector, it was much more work than anticipated. I'm now using Kralik's SAP connector, a very nice piece of software! Now I will start port the Data Warehouse to Azure, this will be interesting eventhough I question the wisdom of the move. The company is writing a replacement for the Data Warehouse. I think it's best just leave my data warehouse as it is, but the company really want to move it to Azure. Many years ago I predicted I would retire before the Data Warehouse, I was right, I was retired years ago. Now I start question myself if I will stop work before the Data Warehouse will be decommisioned. When that happens I will probably ask myself if I will die before the Data warehouse:) Next year 2022 apart from porting the Data Warehouse to Azure (wich is much more than 'lift-and-shift') I will help the DW guys with DW support, and maybe a bit with the DW-successor, I have discussed the Bill-of-Material logic which is a bit complex, and they are under-staffed so they may ask for some help with the . Next year I have promised myself to pick up another (computer) language, I'm interested in Raku and D. Also Svelte and Webassembly looks very interesting. I'm also qurious about Spark. And Yesterday Andreas told me I would enjoy an Oculus headset. So I'm not out of ideas for the future, the question is am I eager enough to start up something new. Time will tell. Anyway apart from peace on Earth I wish you all a happy 2020.

Summer's gone

The Data warehouse lives for another winter. 

Last month I tried to move the database to the Azure cloud. We knew it would probably not work out well, the latency turned out to be a bit over three times longer with the database in the cloud, a bit better than expected, but still to much.

The company is very eager to move the Data Warehouse to the cloud, and I agreed to help them out. It is a lot of work since I have to upgrade software and opsys to modern versions, eg PHP 5 to at least 7.4. Last upgrade of the data warehouse was done 2012. I will most likely work on and off with this project most part of the autumn, eventually into the winter. Right now I have a problem with a closure in PHP 7.4 that drives me nuts. I spent about three days with this and I still do not understand the problem. I think I need to take a step back and really focus on tracing the bug. 

PCHART and PHP 7.4

I have for some time now struggled with PHP 7.4.9 and PCHART, finally this morning I made the combo work. First I started out with PCHART 2.4, I missed the little note saying "This version is not compatible with old PCHART scripts". I probably spent a calendar week trying to make my old scripts work before I started from scratch again, this time I both saw and read the little note. I downgraded PGRAPH to 2.1, still nothing worked. After some debugging I realised '$x = "" ' no longer resulted in $x becoming NULL (undefined) but an empty string, unfortunately that is how PCHART 2.1 NULLIFY/UNDEFINE variables.

But shame on me if I give up coming so far (I seldom give up).

I changed most variants of  ' =" " ' to ' = NULL' and voilà:

There were a lot of ‘ $x = “ “ ‘, and I have probably not changed all of them, but now PCHART 2.1 works for me, and I’m happy for that.

IDRAC to the rescue

 Yesterday was abit hectic, while I was working on some Data Warehouse virtual servers, the database server went bananas, I had lots of server terminals up on my laptop and I shutdown one virtual server to change it's configuration, or so I thought I shutdown the physical host server, all console frames went black and I could not connect to the physical host. I then  to launch the virtual console of the IDRAC, but the connection was dropped probably due to some security policy😠 So there I was at my home without my car (a son had borrowed it for the day), 45 minutes of public transport the hottest day in the year was not very tempting. while pondring on my alternatives I saw the power ON/OFF toggle button on the IDRAC menu. What happens if I press the toggle twice? First power OFF the server and then power ON the server. That should in theory start the server, so I pushed the toggle twice and waited for 15 minutes.

Lo and Behold, the physical host were up and running and so where all virtual servers. Sometimes, very seldom I do admit, things work out exactly as they should. This was one of these rare occations,

Database server running wild

 Yesterday the hottest day so far this summer 32C, the Data Warehouse MySQL server went haywire. Conections where dropped and database tables where partially unavailable this is a very heavy used server no time for deep debugging, so I restarted MySQL and all was normal again, for a few hours, then it really went crazy, now the entire server was affected not only MySQL was uncontactable but the network as well and I could see how new MySQL daemons tried to start up every ten seconds or so. this time I noticed Bacula, ClamAV and Symantec rtvscand was very active at the same time on production prime time, this is probably not a good idea. I never seen anything like this in the twenty years The Data Warehouse been up and running. This time I restarted the database server and now it seems to be OK again.

I have asked the guys responsible for the Data Warehouse to revise the schedules for ClamAV and rtvscan. Why both these daemons are used I do not know. A poolicy of supporters and belts?

Second jab

 At last, I had my second covid jab the day before Yesterday😁 After the first shot I didn't feel any side effects at all. Yesterday I was really sick but today I'm ok again. 

And I'm still working on my old Data warehouse. After all this years. I several times thought I left it behind, but no it is still there hauting me on it's 21 year of operation. I recently found two broken disks in the database raid 6. Not fun, it became  even worse when Dell told me they did not have any replacement disks for the old database server. Luckily I found a company selling refurbished disks. Later I found out Dell did have larger disks perfectly ok as replacements. The Dell representative I spoke to probably didn't know. Anyway now the old Irons are good as new.  

I'm working on reducing wallclock time for the nightly batches and eliminate different kind of errors. Right now I'm looking at posibilities of inject error correction code in the SAP data extraction engine. Normally this kind of processing is done after extracting data from Source systems, but this time it is not possible. A very clever guy has created an entire data warehouse engine inside one one ITL script. Really ingenious, but sort of closed, the extraction of data from SAP into the Data Warehouse is done in one go (actually in 25 parallel independent threads).  

Mars dust buster?

 I have not seen the most obvious question about the Mars helicopter Ingenuity. Will it be used for blowing dust off Perseverance?

That would really be cool, once every now and then Ingenuity takes to the air over Perseverance and blow off the dust.  At home I start my Neato each week to clean my apartment, but for Perseverance once or twice a year would probably be sufficient. 

Update: I just found out Ingenuity is only a PoC with a very limited life expectancy, so it will not be the first dust buster on Mars😟

Data Warehouse and winter

 Last week I had a call from the company they have performance problems with the Data Warehouse. It seems they want me to help them to fix the problems. While they make up their mind I’m enjoying outdoor winter activities.

After a dreadful November, December and January, no sun, slush and rain, came beautiful February. Lots of snow and proper winter temperature down to -10C. Even though the sky looks cloudy we had sun the entire Yesterday. And for the first time this year you felt the warmth of the sun, this is the best time of the year in Stockholm if you ask me.