MySQL Forums
Forum List  »  Replication

Slave Hangs on read lock
Posted by: Mark Lichtenberg
Date: May 13, 2014 10:46AM

Hi,
We're running MySQL Community Ver 14.14 Distrib 5.6.17 using row-based master/slave replication on Amazon's EC2 platform. Recently our slave has been getting hung up (replication stops, slave mysql won't cleanly shutdown) a few times a day when our snapshotting backup (ec2-consistent-snapshot) script runs, and I believe it's due to the script calling FLUSH LOCAL TABLES WITH READ LOCK which gets hung. SHOW PROCESSLIST shows the system user in a state of "Waiting for commit lock" - until I kill this process and restart, any calls to 'show slave status', 'stop slave' or the init shutdown script just hang and have to be cancelled.

This seems similar to the bug http://bugs.mysql.com/bug.php?id=68460, however it looks like it was fixed in 5.6.13, and we're on 5.6.17. The hanging processes started a few days ago when we started experiencing heavier-than-normal load on our master due to a migration script. I believe this load is contributing to the frequency of the hangups since prior to this we've only seen replication stop very infrequently. Our slave cnf config is below. Any help diagnosing this would be much appreciated!

----------------

[mysqld]
port=3306
socket=/var/run/mysqld/mysqld.sock
datadir="/data/mysql"
lower_case_table_names=1
default-storage-engine=INNODB
max_connections=400
query_cache_size=176M
table_open_cache=1520
tmp_table_size=63M
binlog-format=ROW
event_scheduler=ON
tmpdir=/data/mysql/temp

long_query_time=5
slow_query_log
log-queries-not-using-indexes

innodb_additional_mem_pool_size=11M
innodb_flush_log_at_trx_commit=1
innodb_support_xa=1
innodb_log_buffer_size=10M
innodb_buffer_pool_size=5000M
innodb_log_file_size=107M
innodb_thread_concurrency=10
innodb_file_per_table
innodb_autoextend_increment=200M
innodb_data_home_dir = "/data/mysql"
innodb_log_group_home_dir = "/data/mysql"
innodb_log_files_in_group = 2
max_allowed_packet=500M

#replication stuff
server-id=20
log-bin=1
show-slave-auth-info=1
expire_logs_days=7

#####################
# replica stuff
#####################
relay_log=/data/mysql/relay-bin
# stops slave threads from starting after a crash. Must restart slave process manually!
log_slave_updates
skip_slave_start
read_only

Options: ReplyQuote


Subject
Views
Written By
Posted
Slave Hangs on read lock
1804
May 13, 2014 10:46AM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.