Upgrade MySql from 5.6 to 5.7

When I first created The Data Warehouse I envisioned it to be a “poor man’s Data Warehouse”,

due to the small budget I had. I expected it to be good and well working but nevertheless on the cheap side.

But it didn’t take long before I realised I could build a top class Data Warehouse on scrapped hardware and

Mandrake Linux. Already after a year I realised that the prototype I had build was in most aspects better than

anything on the market. At that time MySQL MyIsam storage engine was a dream for the Data Warehouse

database, primitive yes, but had all features we needed and it was fast, very fast on the scrapped PCs we run it on.

Easy to copy and backup, actually we didn’t do any database administrative tasks at all apart from backups.

Ulf the first developer never took any notice of my warnings “You must pay attention to indexes they are very important.”  

“I have no time for your diddle-daddle I add single column indexes I know will be used and that’s it.” he replied.

And that’s basically the database administrative we have done now for eighteen years. (There have been

a few cases where we have been forced to create composite indexes.)

We do not have a test environment on purpose, when needed we create a test database on the production

database server. The single production environment is one of the key success factors for the Data Warehouse,

it cuts down development and debugging time a lot. There is a slight drawback with this approach as we never

clean out rubbish and not used tables we have piled up a lot of ‘dead’ databases and tables over the years.

It is much like DNA The Data Warehouse evolves tables are replace or get obsolete, test database are abandoned etc

Of the 1 terabyte data in the Data Warehouse about 30 to 60 percent is not used or rubbish that should not

be used. But who cares it’s just disk space and bigger backups. We do not waste time on administration

we maintain and develop new apps 100 percent of the working hours.  The waste do not do much harm,

or rather up until now, we are now preparing to upgrade MySql from version 5.6 to 5.7, and a test migration took +30 hours,

The mysql-upgrade script just never stopped or so it seemed. More about that in the next post.