MySQL 8.4.3 – Transactions stuck in waiting for handler commit until restart (async master–master replication, high concurrency on InnoDB)

[kayukaran Parameswaran]

We are experiencing an intermittent but severe commit stall in MySQL 8.4.3 Community Edition (RHEL-compatible OS) running asynchronous master–master replication.

Symptoms:

Many client sessions remain in waiting for handler commit for thousands of seconds.

Affected transactions are short (single-row UPDATE/INSERT/DELETE on InnoDB table radius_auth.re_auth_ctx).

Application connections start aborting with Got an error reading communication packets.

Condition persists until mysqld is restarted. After restart, same workload runs normally.

The same workload on MySQL 5.7.15 does not show this behavior.

Replication thread (SHOW PROCESSLIST) is idle — "Source has sent all binlog to replica; waiting for more updates".

InnoDB redo log status shows no backlog; LSN positions are stable and aligned.

No Group Replication — only classic async replication.

Example stuck sessions:


| 31098 | sdpuser | ... | radius_auth | Query | 7651 | waiting for handler commit | update re_auth_ctx set next_re_auth_id=... |
| 31100 | sdpuser | ... | radius_auth | Query | 7651 | waiting for handler commit | update re_auth_ctx set next_re_auth_id=... |
Table DDL:


CREATE TABLE re_auth_ctx (
correlation_id varchar(50) NOT NULL,
protocol enum('SIM','AKA') NOT NULL,
imsi bigint NOT NULL,
updated timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
created timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
identity varchar(200) DEFAULT NULL,
next_re_auth_id varchar(200) DEFAULT NULL,
counter int NOT NULL DEFAULT '0',
master_key varchar(100) DEFAULT NULL,
k_aut varchar(100) DEFAULT NULL,
k_encr varchar(100) DEFAULT NULL,
nonce_s varchar(100) DEFAULT NULL,
PRIMARY KEY (imsi)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
/*!50100 PARTITION BY HASH (imsi) PARTITIONS 100 */
Workload:

High QPS from multiple application nodes (RADIUS re-auth).

Each transaction updates or inserts a single PK row.

Async master–master replication between two MySQL 8.4.3 servers; only one server exhibits this issue at a time.

my.cnf highlights:

makefile
Copy
Edit
innodb_flush_log_at_trx_commit=1
innodb_doublewrite=0
innodb_flush_method=O_DIRECT
innodb_log_file_size=2G
innodb_buffer_pool_size=8G
max_connections=500
binlog enabled (async replication)


Additional Notes:

Would appreciate guidance on whether this matches an existing 8.4.3 issue, and if upgrading to ≥8.4.6 is recommended.