2018-09-20

Checking the database

I’m writing a series of posts about migrating out Data Warehouse Database from MySQL version 5.6 to 5.7
(ultimately 5.8). Yesterday I did a test to check the present database, I wrote a script checking all tables
(little less than 4000) in 10 parallel streams, it did not go well, after some 4 hours and still 2500 tables to
check I cancelled the check tables script. Some tables took more than 30 minutes to medium check.
Maybe I’m checking to many in parallel, anyway I decided not to go for a medium check but a quick check
I test this Sunday night.
<?xml version='1.0' encoding='UTF-8' standalone='yes'?>
<schedule>
  <job name='checkTables'>
     <forevery chunks='10' parallel='yes'>
<sql>SELECT a.table_schema DB,a.table_name TB
   FROM information_schema.tables a
   WHERE  a.TABLE_TYPE = 'BASE TABLE';
</sql>
     </forevery>
     <myiterator piggyback='forevery'/>
     <sql>
CHECK TABLE @DB.@TB MEDIUM;
     </sql>
  </job>
</schedule>


Why do I do this? Before the migrate it is good to know that all tables are ok, but I cannot wait some 10 to
15 hours for the check to finish.

No comments:

Post a Comment