The bug

The bug I blogged about the other day was just too little memory, I missed the error message. The memory_limit was set to 4Gb, I changed that to unlimited  ini_set(‘memory_limit’, -1). One problem less in the world.

Stupid robots in a brave world

This morning I read an interesting article in Dagens Nyheter (News of the Day). The author goes down memory lane of journalism to make a statement on auto generated journalism.  

(Unfortunately the article is written in Swedish, but you can run it through Google translate to enjoy auto-translation to the lingo of your choice). The first newspaper/tabloid in Sweden is now replacing sport journalists with computer software. The author is negative to ‘computer journalism’, he argues real journalism is to publish what you as a journalist wants to write about and some others disapprove of, and that is something the media robots cannot do. This is what you can expect from a journalist and that is not what is interesting with his article. He argues freedom of speech, expression or press will not apply to robots or computer software.

I find this both a bit strange and intriguing. It’s a bit like claiming text produced by a typewriter cannot be a subject of freedom of expression. So far no robot or software has a free will and can do whatever it pleases, there is a human mind behind all these devices and there is an accountable human owner. That is today but in future? We see a new information technology revolution, more intellectual tasks are taken over by computer systems, e.g. human financial advisers are replaced by computers. 

If many more intellectual  tasks are done by computers I can see situations where the accountability is unclear, e.g. a team of software financial advisers after consulting robot solicitors on behalf of clients makes bad or even criminal investments via a computerized bank. Who is accountable? Or the surgeon robot amputating the wrong hand due to unclear advice from the software radiologist. You do not need to have a free will or true intelligence to do things you should be accountable for, computer or human, you only have to be stupid. I can think of far worse things done by software gone astray or wild in the not so distant future, than auto generated journalism.

In principle identical, fundamentally different

The data warehouse is a very robust piece of software, it got it's rough spots and edges, but it hardly ever fails. So it's not surprising Henrik the newest in the line of bright youngsters working in the Data Warehouse have not been exposed to bugs before. But now he found a gronk, a huge SAP data extraction fails silently leaving no traces and no data. I looked at the problem, it looked like a no-brainer, so I just added a few tracing statement in my development environment and everything worked well I got about seven million rows from SAP, so I assumed I in the past had fixed the bug but not moved the module to the production environment. I moved the development module into production but forgot to test, which of course had consequences, some hundred extraction jobs failed that night. We had to roll back the original version of the script and Henrik had to rerun the failing jobs. Later when Henrik discussed the bug with Camilla (the BI team lead) he said "It's a bit puzzling, it works in the development environment and Lars told me it is in principle identical to the production environment." Camilla burst into laughter "when Lars says 'in principle identical', he means fundamentally different".

I have had another look into this bug and it looks like we have a problem with the str_replace function of PHP. With a bit of luck I will hunt this bug down during the weekend. I have promised not to touch the production environment though.

Many years from now, some years after I left the company the first time I had a chat with the IT manager and he said "You are great and you have built us the best production environment for our ERP systems in the world, but the environment have stabilized a lot since you left".

It is very hard to combine innovative, stable and low cost. You can pick any two, but you cannot pick all three. When I started the Data Warehouse project I decided to go for innovative and low cost.The first because that’s what I am, the second out of necessity, I didn’t have any funds. But still the Data Warehouse is remarkable stable as long as I do not interfere.

The log comes in colors like a rainbow

The log of the data warehouse ETL engine has been black and white up until now. Some years ago I added colors to the log. However the same text string printed to console was also written to the archive log where the color attributes looked awful. But a few weeks ago while jogging an early morning I got the idea of filter out all attributes with a regular  expression.

This is the original code writing the console log message to the archive log

fwrite($this->logHandle, "$lmsg");

This regular expression sifts out all color attributes:

fwrite($this->logHandle, preg_replace('/\033\[[\d;]+m/', '',"$lmsg"));

Simple as that:)

This is a black and white example of the console log:

And this is the color version:

This is not the most useful hack I have done and it isn't exactly a rainbow, but the console log is easier to read.

Can you explain me

As I have written before, the 'company language' is English. This means the most used language is funny English since only a few of us has English as the mother tongue. I both write and speak funny English as I'm doing right now. Funny English is a language with many dialects. One grammar rule I think stems from central Europe (German,Flemish,Dutch) is to omit the introductory 'to' when you have a transitive with an indirect object. E.g. in proper English 'can you explain to me ...' becomes 'can you explain me ...'.
To me as a Swede omitting 'to' sounds weird, but it has become so common at the company I have heard native English speakers yield to this funny English grammar rule. I suppose this is one way languages evolve. I'll bet my 5 cent the introductory 'to'  will disappear in English in the next 100 years.

Good reading for programmers

Once in a while you stumble upon a great post, this is one http://www.2ality.com/2015/03/es6-generators.html?m=1

Do not edit!

Sometimes I feel things get overworked.

The other day I did a big mistake when upgrading an Ubuntu server running PhpMyAdmin from version12.04 to 14.04. I got the question ‘Do you want to keep the PhpMyAdmin config or replace it with a new config?’. I replied ‘replace with new’, that was almost kissing PhpMyAdmin goodbye.

As I recalled it PhpMyAdmin config was a simple file taking a few minutes to fill in and then just restart apache. But not so anymore, in Ubuntu it seems PhpMyadmin is scattered all over. I found several config.inc.php files and I tried to configure PhpMyAdmin by updating them one at a time but none of these updates worked. Finally I gave up and edited the file /usr/share/phpmyadmin/libraries/config.default.php. You see the header of the config.default.php file above.

I do not see the problem of having one configure file, in fact that is as it’s supposed to be, one simple text file, where you enter simple configure declaratives to adopt the software to your environment and liking. Splitting a simple configuration file into many just makes the configure process harder not simpler. To me it seems someone forgot was configure is about, and went over the top to create the perfect configure process. Often I found less is more.