2013-05-31

PHP 5.5 and Mageia 2

I have written some posts  about moving our Business Intelligence solution The Data warehouse to new hardware. We also upgrade the software and reshuffle some components, actually its like building a new system. The ETL server is now virtualized from Mandriva 2010 Linux to Mageia 2 Linux as we could not get tightVNC to work in Mageia 3! It seems like the history repeats itself from the Mandrake days, every second version has issues. Anyway Mageia 2 runs nicely so far, touch wood. When Mandrake, Mandriva, Mageia works these distros are second to none. So the ‘brain’ of the data warehouse runs on Mageia these days. With no problem at all I compiled PHP 5.5 RC2  together with the SAP rfc communications modules SAPRFC  and SAPNWRFC . So we soon run a serious large scale production, (some 20,000 jobs a month), on Mageia 2 and PHP 5.5 RC2 - I love it. Now I have to come up with a way to upgrade MySQL from 5.5 to 5.6, we have a tricky Mysql setup , but I will find a way to do it.  (In Mageia I run MariaDB Client against MySQL 5.5 at first I didn’t even notice).  

This whole project revives the original spirit of the data warehouse project, new innovative and low budget. Year 2000 I set out on a journey to prove you can create high quality IT systems on a shoestring budget. I think I done it again. This time it’s way more expensive, my bosses forced me to spend much more money than I had in mind, how often does that happen? Still we have a very modest budget considering the application.          

2013-05-16

Moving a Data Warehouse - 1

We have moved our Data Warehouse database from my old server  to a Dell R720 .
 
From Mandriva 2010 MySQL 5.5.9, to Ubuntu 12.04 LTS MySQL 5.5.31.
We copied the 400GB Database with Rsync. It took some 7 hours to migrate, but after that it was just to fire up mySQL in the new server and change DNS addresses to point to the new server. In the old server we set up a MySQL proxy to reroute inbound traffic to the new server just in case. The transition was extremely smooth not a single problem until a mail arrived
‘Is there a problem with the Data Warehouse? ’  
When our application calls the Data Warehouse we receive   ‘4.0 protocol is not supported ’, please help.
WTF, this was not something I expected by upgrading MySQL some patch levels of the same release. After we have rummaged the building we found an old derelict Dell server in the basement. It turned out to be an old Mandrake 8.1 Linux running on the machine for years, and those who knew of the server setup had long since left. We (wisely) decided not to touch this server, instead we started up a copy of the database in version 5.5.9 and rerouted the obsolete application(s) to this DB instance. Next we will virtualize a copy of this Linux system and then upgrade it to more modern Linux probably a Mageia Linux.    
 
 

2013-05-03

New Data Warehouse Servers

I have written some posts about design and build your own BI infrastructure . Now when I’m getting a new assignment, we (my bosses :-) decided I had to replace my hardware  with more professional irons.

The new Data Warehouse

I chose two servers from Dell, o ne PowerEdge R720  and one R720XD , both equipped with one Xeon ES-2609, 32GB 1600MHz RAM and 12TB RAID 6 SATA/Nearline SAS disk space ( I like oceans of space ).

I will use the R720 as a physical MySQL database server , and the R720XD will contain all other (virtual) servers. Going virtual is an architectural change I planned to do for a long time, but I never found the time. I rather would have used my custom build hardware, but the guys who decide decided otherwise. ‘Since you jump ship, we do not dare to use your old computers . We want proper servers.’ I do not know how the new infrastructure will perform but I got a hunch it will perform better than my now old irons. I will also go from 100MB network connections to gigabit, this will definitely speed up communications.