2012-05-26

Replicate Business Intelligence Data across the globe

Working in a multinational company you sometimes end up in places you never dreamed of. I’m writing this from  a hotel room in Osaka; Japan. We have a factory here, and we are implementing SAP for purchasing, logistics and production planning, and accompanying SAP is our Business Intelligence system the Data Warehouse, (which is my responsibility). When testing response times from the Osaka plant ‘Fuji Air Tools’ half a year ago , the response times were slow. I was asked if we could do something to speed up response times. I hope this is  easy, we just install a local Data Warehouse server replica in the Osaka plant. Any decent workstation PC  will do since we will just replicate some MySql databases over to Osaka. My plan is to use a very simple Rsync replication scheme. Each day when time is due we just offline the ‘Osaka database ‘ and rsync it from the master in Stockholm.
Why not set up real time replication? I tried this a few times, but this is more complicated. Live replication must be monitored, and it inevitably will fail at some time in a way manual intervention is required. My Rsync replication is simple and self-healing if it which God forbid should fail. (Just rerun the rsync.)
I hear people say you cannot backup/copy/replicate a live database like that. You must at least flush the tables and block updates of the master during Rsync, otherwise you will have a corrupt replica. But I will Rsync until there is no differences between the replica and the master. This is a BI system and updates are mostly controlled background  jobs. My Rsync scheme will work well.

Doers and Thinkers.

I am of the opinion BI systems should be on Local  Area Networks not Wide  Area Networks. BI systems shuffle lot of data, response times will always be better in a LAN, and the band width can be used better than repeatedly send BI analysis queries, this is what the thinkers  do. What happens if I change… and then the thinker resubmit a similar query once again and thinkers result tables are often big.
You can divide Business Intelligence users into Doers  and Thinkers . Doers logs on early in the morning and use the BI system for their daily job. They run their reports, e.g. in a factory production planners and logisticians may analyze component shortages and  adjust today's production accordingly. Thinkers on the other hand come in later and work with issues like ‘What happen if we build a new plant in...’ . And they tend to not only browse through lot of data they also pick up a lot of data.

Scared of airports, not feared of flying.

This is not at all what I intended to write. I intended to write about my adventures in airports, with the title scared of airports not of flying. Coming to an airport is like descend into hell, I probably have a pulse rate like a soldier in combat. For me everything goes wrong, this time I started by losing my boarding cards. It is not until I’m on the airplane I can calm down.
You know these self-checking machines where you are supposed to type in your booking number, e-ticket number or whatever. I NEVER type the right number, I type the forms number, the date or something that these stupid machines do not recognize as me.
I do not like airports, it seems to be genetic .

No comments:

Post a Comment