MySQL Forums
Forum List  »  Replication

Extremely high disk IO when replication is enabled
Posted by: William Wilson
Date: May 18, 2018 04:24PM

We've been migrating from MySQL 5.1 on a virtual machine (which performed mostly fine) to two servers running MySQL 8.0.4-rc on high-performance dedicated hardware. The two new servers replicate to each other, but only one has a floating IP and is writable at a time (master-master active-passive).

The problem is, we're seeing an insane amount of disk IO on both of these new servers, between 500 MB/s and 1300 MB/s.

We suspect the new replication setup is somehow causing the heavy IO, because when I temporarily shut down the passive MySQL server, disk IO drops to what we would expect on both servers.

I'm wondering if we've either misconfigured something or encountered a bug. Obviously there are a lot of changes going on here with the new setup and new version, but I'm wondering if there is any explanation for the high IO. Here is the output:

database 1:

# iostat -m 5
...
avg-cpu: %user %nice %system %iowait %steal %idle
1.59 0.00 0.69 0.00 0.00 97.72

Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 9896.00 0.00 1235.24 0 6176
dm-0 0.00 0.00 0.00 0 0
dm-1 0.00 0.00 0.00 0 0
dm-2 9896.00 0.00 1235.14 0 6175

avg-cpu: %user %nice %system %iowait %steal %idle
1.29 0.00 0.63 0.01 0.00 98.08

Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 7188.62 0.00 899.06 0 4504
dm-0 0.00 0.00 0.00 0 0
dm-1 0.00 0.00 0.00 0 0
dm-2 7189.22 0.00 899.11 0 4504

avg-cpu: %user %nice %system %iowait %steal %idle
1.95 0.00 0.67 0.00 0.00 97.38

Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 8825.00 0.00 1110.22 0 5551
dm-0 1.00 0.00 0.06 0 0
dm-1 0.00 0.00 0.00 0 0
dm-2 8824.60 0.00 1110.16 0 5550


database 2:

# iostat -m 5
...
avg-cpu: %user %nice %system %iowait %steal %idle
0.45 0.00 0.73 0.10 0.00 98.72

Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 11002.99 0.00 1398.01 0 7004
dm-0 0.00 0.00 0.00 0 0
dm-1 11002.99 0.00 1398.01 0 7004

avg-cpu: %user %nice %system %iowait %steal %idle
0.56 0.00 0.73 0.10 0.00 98.61

Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 11028.20 0.00 1377.61 0 6888
dm-0 0.00 0.00 0.00 0 0
dm-1 11028.20 0.00 1377.61 0 6888

avg-cpu: %user %nice %system %iowait %steal %idle
0.62 0.00 0.76 0.11 0.00 98.51

Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 11069.80 0.01 1385.97 0 6929
dm-0 1.00 0.00 0.05 0 0
dm-1 11069.20 0.01 1385.92 0 6929

Options: ReplyQuote


Subject
Views
Written By
Posted
Extremely high disk IO when replication is enabled
1064
May 18, 2018 04:24PM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.