2015-05-24

Mobile Business Intelligence

The Data Warehouse is conceptually divided into one ETL backend, and frontend viewers. One of those viewers are Qlikview. We run Qlikview as web apps. The Qlikview web environment is device sensitive or adoptable, which means that you can run QV apps on a 27 inch hi resolution screen or a smartphone, you just need a web browser. Of course the user experience is much better on a 27 inch screen, but Qlikview does a good job of right scaling the app. We had the infrastructure for mobile BI in place for years but little or no interest from the users. But this is changing, lately mobile BI has got a lot of traction in the company.
How useful a BI app in a smartphone is remains to be seen. I feel it is more a gimmick than serving a useful purpose, the screen is too small. But for a dexterous person with small hands it might come handy on the road. My prediction is most likely wrong, in just a few years we all will run a lot of apps in our smartphones actually I’m doing it already. One thing that surprises me is how performant the Qlikview apps are in my smartphone, with 4G I have really good response times.

Security is a big concern for mobile BI.

As you can see below I pixelated parts of a smartphone BI screenshot, BI information is trade secrets, I have chosen a ‘harmless’ app, but I hide some of the information just to be on the safe side. BI is to analyse and compare the business, going outside of the firewall with this kind of information is not without risks. Curious guys are on the net fishing for information. National organisations like NSA, Unit 61398 or FRA, all such organisations are probably involved in industrial espionage to some extent, not to mention competitors and criminal organisations. You should protect BI information as the confidential business assets they are. Going mobile is a risk, and you should be aware of that.

A Qlikview small device screenshot:

Mobile BI - in action on my Samsung Galaxy S4

2015-05-19

20:00 still at the office


I’m sitting alone in the office burning midnight oil got to finish a report, listening to Peter Green. Although I’m an atheist it struck me that Mr. Green should at least be part of the entertainment in the paradise. The music he creates is divine, pure magic.  

2015-05-18

Mini vacation and meetings

Back to the office after Ascension Day mini-vacation. Beginning of last week I was in Örebro, discussing Master Data management with IT-architect colleagues and then two days free. But now I'm back again with a filled calendar and a lot of new meeting invites in the mail. I'm not a 'meeting-guy', more than two, three hours meeting a day is mental torture according to me. I have a feeling meeting getting more frequent by the year, these days all seems to be in meetings most of the work hours, and a lot of other hours as well, if you have telcos with people in Europe, US, Australia and China you sometime have to work at very odd hours. Anyway this week there is a lot of meetings for me.
Last week I discovered Canary Pete he is hilarious, he has got the essence of what meetings is all about:

2015-05-16

RTFM, PHP closures revisited

Stockholm 2015-05-16


Some time ago I wrote two posts called Dynamic PHP code by closures and Dynamic PHP code 2. I complained about having to use the eval instruction when I created the closure, this was of course wrong. Now when I need to create another closure from a text script, I decided to read the manual. Instead of  using the function instruction as used in my previous attempt, I found out that create_function allows me to do what I want without eval. So without further ado:


Sometimes the SAP formats values a bit odd e.g. instead of responding 100 for one hundred percent SAP gives *0.00, this is not a value you can import in a numeric declared database column. To solve this and other formatting problems I decided to implement post formatting as PHP closures, an example:
This job executes the SAP bapi BAPI_CURRENCY_GETLIST. If you look at the <columnfunctions> You can see I added formatting functions for three columns; LONG_TEXT,CURRENCY and MESSAGE. 

The PHP code creating the closures from this looks like:
Running the job above results in the following MySQL table:

2015-05-06

The bug


The bug I blogged about the other day was just too little memory, I missed the error message. The memory_limit was set to 4Gb, I changed that to unlimited  ini_set(‘memory_limit’, -1). One problem less in the world.

2015-05-04

Stupid robots in a brave world

This morning I read an interesting article in Dagens Nyheter (News of the Day). The author goes down memory lane of journalism to make a statement on auto generated journalism.  
(Unfortunately the article is written in Swedish, but you can run it through Google translate to enjoy auto-translation to the lingo of your choice). The first newspaper/tabloid in Sweden is now replacing sport journalists with computer software. The author is negative to ‘computer journalism’, he argues real journalism is to publish what you as a journalist wants to write about and some others disapprove of, and that is something the media robots cannot do. This is what you can expect from a journalist and that is not what is interesting with his article. He argues freedom of speech, expression or press will not apply to robots or computer software.
I find this both a bit strange and intriguing. It’s a bit like claiming text produced by a typewriter cannot be a subject of freedom of expression. So far no robot or software has a free will and can do whatever it pleases, there is a human mind behind all these devices and there is an accountable human owner. That is today but in future? We see a new information technology revolution, more intellectual tasks are taken over by computer systems, e.g. human financial advisers are replaced by computers. 
If many more intellectual  tasks are done by computers I can see situations where the accountability is unclear, e.g. a team of software financial advisers after consulting robot solicitors on behalf of clients makes bad or even criminal investments via a computerized bank. Who is accountable? Or the surgeon robot amputating the wrong hand due to unclear advice from the software radiologist. You do not need to have a free will or true intelligence to do things you should be accountable for, computer or human, you only have to be stupid. I can think of far worse things done by software gone astray or wild in the not so distant future, than auto generated journalism.

2015-05-02

In principle identical, fundamentally different

The data warehouse is a very robust piece of software, it got it's rough spots and edges, but it hardly ever fails. So it's not surprising Henrik the newest in the line of bright youngsters working in the Data Warehouse have not been exposed to bugs before. But now he found a gronk, a huge SAP data extraction fails silently leaving no traces and no data. I looked at the problem, it looked like a no-brainer, so I just added a few tracing statement in my development environment and everything worked well I got about seven million rows from SAP, so I assumed I in the past had fixed the bug but not moved the module to the production environment. I moved the development module into production but forgot to test, which of course had consequences, some hundred extraction jobs failed that night. We had to roll back the original version of the script and Henrik had to rerun the failing jobs. Later when Henrik discussed the bug with Camilla (the BI team lead) he said "It's a bit puzzling, it works in the development environment and Lars told me it is in principle identical to the production environment." Camilla burst into laughter "when Lars says 'in principle identical', he means fundamentally different".
I have had another look into this bug and it looks like we have a problem with the str_replace function of PHP. With a bit of luck I will hunt this bug down during the weekend. I have promised not to touch the production environment though.

Many years from now, some years after I left the company the first time I had a chat with the IT manager and he said "You are great and you have built us the best production environment for our ERP systems in the world, but the environment have stabilized a lot since you left".
It is very hard to combine innovative, stable and low cost. You can pick any two, but you cannot pick all three. When I started the Data Warehouse project I decided to go for innovative and low cost.The first because that’s what I am, the second out of necessity, I didn’t have any funds. But still the Data Warehouse is remarkable stable as long as I do not interfere.