Re: MySQL 8.0.16 on Fedora/PPC64LE : I see issues, inc. hangs

[Tony Reix]

Thanks for the recommendations.
About building MySQL on my Fedora/PPC64LE machine:
- I've created a "mysql" user and I've logged as: su - mysql .
- I'm using the community-mysql-8.0.16-2.fc31.spec RPM .spec file from Fedora
I've made some NOT-important changes to this file: cmake instructions are exactly the same.
- so, I'm at: /var/lib/mysql/rpmbuild/SPECS
- and RPM creates:
/var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16
/var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/build
automatically
- I'm using my own Test script based on what appears in the Fedora .spec file but seemed to me not fully correct:
cd ../BUILD/mysql-8.0.16
cd build
cd mysql-test
export MTR_BUILD_THREAD=64

./mtr \
--verbose \
--parallel=1 \
--force \
--retry=0 \
--mysqld=--binlog-format=mixed \
--suite-timeout=2000 \
--testcase-timeout=60 \
--report-unstable-tests \
--max-save-core=100 \
--max-test-fail=0 \
--vardir=/tmp/mysql-notroot \
$@

I've applied your changes to: /etc/sysctl.conf & /etc/security/limits.conf and I followed your instructions, and I rerun the innodb.blob-crash-4k test.
- First, the test passed OK
- But, on 2nd try, it hanged again:
top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
11170 mysql 20 0 1536128 87808 35264 S 32,6 0,5 2:12.78 mysqld

So, the hang is random.

# gdb /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/build/runtime_output_directory/mysqld
...
(gdb) attach 11170
(gdb) i th
Id Target Id Frame
* 1 Thread 0x7fffb84baee0 (LWP 11170) "mysqld" 0x00007fffb83fa454 in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
2 Thread 0x7fffb4ddf060 (LWP 11171) "mysqld" 0x00007fffb840186c in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
3 Thread 0x7fffaffff060 (LWP 11173) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
4 Thread 0x7fffad8af060 (LWP 11174) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
5 Thread 0x7fffad09f060 (LWP 11175) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
6 Thread 0x7fffac88f060 (LWP 11176) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
7 Thread 0x7fff9ffff060 (LWP 11177) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
8 Thread 0x7fff9f7ef060 (LWP 11178) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
9 Thread 0x7fff9efdf060 (LWP 11179) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
10 Thread 0x7fff9e7cf060 (LWP 11180) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
11 Thread 0x7fff9dfbf060 (LWP 11181) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
12 Thread 0x7fff9d7af060 (LWP 11182) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
13 Thread 0x7fff9cf9f060 (LWP 11183) "mysqld" 0x00007fffb840186c in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
14 Thread 0x7fff797cf060 (LWP 11190) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
15 Thread 0x7fff727cf060 (LWP 11191) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
16 Thread 0x7fff78fbf060 (LWP 11192) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
17 Thread 0x7fff737ef060 (LWP 11193) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
18 Thread 0x7fff73fff060 (LWP 11194) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
19 Thread 0x7fff72fdf060 (LWP 11195) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0

I got again exactly the same issue I've seen already 5 times.
Here below is the description of the main threads:

Attaching to program: /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/build/runtime_output_directory/mysqld-debug, process 11170
[New LWP 11171]
[New LWP 11173]
[New LWP 11174]
[New LWP 11175]
[New LWP 11176]
[New LWP 11177]
[New LWP 11178]
[New LWP 11179]
[New LWP 11180]
[New LWP 11181]
[New LWP 11182]
[New LWP 11183]
[New LWP 11190]
[New LWP 11191]
[New LWP 11192]
[New LWP 11193]
[New LWP 11194]
[New LWP 11195]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
0x00007fffb83fa454 in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.29-12.fc30.ppc64le glibc-2.29-15.fc30.ppc64le libaio-0.3.111-4.fc30.ppc64le libevent-2.1.8-5.fc30.ppc64le libgcc-9.1.1-1.fc30.ppc64le libicu-63.2-2.fc30.ppc64le libstdc++-9.1.1-1.fc30.ppc64le libxcrypt-4.4.6-2.fc30.ppc64le lz4-libs-1.8.3-2.fc30.ppc64le numactl-libs-2.0.12-2.fc30.ppc64le openssl-libs-1.1.1c-2.fc30.ppc64le protobuf-lite-3.6.1-3.fc30.ppc64le zlib-1.2.11-16.fc30.ppc64le
(gdb) i th
Id Target Id Frame
* 1 Thread 0x7fffb84baee0 (LWP 11170) "mysqld" 0x00007fffb83fa454 in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
2 Thread 0x7fffb4ddf060 (LWP 11171) "mysqld" 0x00007fffb840186c in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
3 Thread 0x7fffaffff060 (LWP 11173) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
4 Thread 0x7fffad8af060 (LWP 11174) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
5 Thread 0x7fffad09f060 (LWP 11175) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
6 Thread 0x7fffac88f060 (LWP 11176) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
7 Thread 0x7fff9ffff060 (LWP 11177) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
8 Thread 0x7fff9f7ef060 (LWP 11178) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
9 Thread 0x7fff9efdf060 (LWP 11179) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
10 Thread 0x7fff9e7cf060 (LWP 11180) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
11 Thread 0x7fff9dfbf060 (LWP 11181) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
12 Thread 0x7fff9d7af060 (LWP 11182) "mysqld" 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
13 Thread 0x7fff9cf9f060 (LWP 11183) "mysqld" 0x00007fffb840186c in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
14 Thread 0x7fff797cf060 (LWP 11190) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
15 Thread 0x7fff727cf060 (LWP 11191) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
16 Thread 0x7fff78fbf060 (LWP 11192) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
17 Thread 0x7fff737ef060 (LWP 11193) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
18 Thread 0x7fff73fff060 (LWP 11194) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
19 Thread 0x7fff72fdf060 (LWP 11195) "mysqld" 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
(gdb) thread 1
[Switching to thread 1 (Thread 0x7fffb84baee0 (LWP 11170))]
#0 0x00007fffb83fa454 in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
(gdb) where
#0 0x00007fffb83fa454 in __pthread_timedjoin_ex () from /lib64/libpthread.so.0
#1 0x00007fffb83fa170 in pthread_join () from /lib64/libpthread.so.0
#2 0x00000001154be234 in my_thread_join (thread=<optimized out>, value_ptr=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/mysys/my_thread.cc:100
#3 0x00000001141d28ac in bootstrap::run_bootstrap_thread (file=0x0, boot_handler=<optimized out>, thread_type=<optimized out>)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/sql/bootstrap.cc:423
#4 0x0000000115130524 in dd::Dictionary_impl::init (dd_init=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/sql/dd/impl/dictionary_impl.cc:146
#5 0x000000011512ad90 in dd::init (dd_init=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/sql/dd/impl/dd.cc:65
#6 0x0000000113db88d4 in init_server_components () at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/sql/mysqld.cc:5327
#7 0x0000000113dbea38 in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/sql/mysqld.cc:6305
#8 0x0000000113d6c880 in main (argc=<optimized out>, argv=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/sql/main.cc:25
(gdb) thread 2
[Switching to thread 2 (Thread 0x7fffb4ddf060 (LWP 11171))]
#0 0x00007fffb840186c in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
(gdb) where
#0 0x00007fffb840186c in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
#1 0x00000001156968b4 in os_event::wait (this=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/include/sync0types.h:536
#2 os_event::wait_low (this=0x7fffb000b5f8, reset_sig_count=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/os/os0event.cc:337
#3 0x0000000115696ef0 in os_event_wait_low (event=<optimized out>, reset_sig_count=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/os/os0event.cc:523
#4 0x00000001157f2a60 in rw_lock_debug_mutex_enter () at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/sync/sync0debug.cc:1188
#5 0x00000001157e13d4 in rw_lock_own (lock=0x7fffa84ca290, lock_type=4) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/sync/sync0rw.cc:957
#6 0x00000001158f7404 in buf_pool_watch_is_sentinel (buf_pool=0x7fffb0159ef8, bpage=0x7fffadb56760)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/include/buf0types.h:183
#7 0x0000000115925c90 in buf_page_hash_get_locked (buf_pool=buf_pool@entry=0x7fffb0159ef8, page_id=..., lock=0x7fffb4dd4870, lock@entry=0x7fffb4dd4970, watch=false, lock_mode=4)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/include/buf0buf.ic:1016
#8 0x000000011592ad20 in buf_flush_try_neighbors (n_to_flush=18446744073709551614, n_flushed=<optimized out>, flush_type=<optimized out>, page_id=<synthetic pointer>...)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/buf/buf0flu.cc:1529
#9 buf_flush_page_and_try_neighbors (bpage=bpage@entry=0x7fffadb56760, flush_type=flush_type@entry=BUF_FLUSH_LIST, n_to_flush=n_to_flush@entry=18446744073709551614,
count=count@entry=0x7fffb4dd49f8) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/buf/buf0flu.cc:1633
#10 0x000000011592c07c in buf_do_flush_list_batch (lsn_limit=9223372036854775807, min_n=18446744073709551614, buf_pool=0x7fffb0159ef8)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/buf/buf0flu.cc:1860
#11 buf_flush_batch (lsn_limit=9223372036854775807, min_n=18446744073709551614, flush_type=BUF_FLUSH_LIST, buf_pool=0x7fffb0159ef8)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/buf/buf0flu.cc:1925
#12 buf_flush_do_batch (buf_pool=0x7fffb0159ef8, type=<optimized out>, min_n=18446744073709551614, lsn_limit=9223372036854775807, n_processed=0x7fffb4dd4ae0)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/buf/buf0flu.cc:2052
#13 0x000000011592cb74 in buf_flush_lists (min_n=18446744073709551614, lsn_limit=9223372036854775807, n_processed=0x7fffb4dd4b70)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/buf/buf0flu.cc:2102
#14 0x000000011592cce4 in buf_flush_sync_all_buf_pools () at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/buf/buf0flu.cc:3333
#15 0x000000011563b188 in log_preflush_pool_modified_pages (log=..., new_oldest=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/log/log0chkp.cc:709
#16 0x000000011563ca94 in log_make_latest_checkpoint (log=...) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/log/log0chkp.cc:669
#17 0x000000011563cccc in log_make_latest_checkpoint () at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/log/log0chkp.cc:688
#18 0x000000011591ed94 in buf_dblwr_create () at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/buf/buf0dblwr.cc:300
#19 0x00000001157d1f90 in srv_start (create_new_db=<optimized out>, scan_directories=...) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/srv/srv0start.cc:2681
#20 0x0000000115560474 in innobase_init_files (tablespaces=0x7fffb4dde358, dict_init_mode=DICT_INIT_CREATE_FILES)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/handler/ha_innodb.cc:4994
#21 innobase_ddse_dict_init (dict_init_mode=<optimized out>, version=<optimized out>, tables=0x7fffb4dde340, tablespaces=0x7fffb4dde358)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/handler/ha_innodb.cc:11657
#22 0x0000000115157d1c in dd::bootstrap::DDSE_dict_init (thd=<optimized out>, dict_init_mode=<optimized out>, version=<optimized out>)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/sql/dd/impl/bootstrap/bootstrapper.cc:744
#23 0x000000011515cd50 in dd::bootstrap::initialize (thd=0x1001ea2f3d0) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/sql/dd/impl/bootstrap/bootstrapper.cc:898
#24 0x00000001141d2344 in bootstrap::handle_bootstrap (arg=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/sql/bootstrap.cc:347
#25 0x0000000115b984bc in pfs_spawn_thread (arg=0x1001e9bab40) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/perfschema/pfs.cc:2836
#26 0x00007fffb83f8c10 in start_thread () from /lib64/libpthread.so.0
#27 0x00007fffb576d8a8 in clone () from /lib64/libc.so.6
(gdb) thread 3
[Switching to thread 3 (Thread 0x7fffaffff060 (LWP 11173))]
#0 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
(gdb) where
#0 0x00007fffb57660b4 in syscall () from /lib64/libc.so.6
#1 0x00007fffb6380888 in ?? () from /lib64/libaio.so.1
#2 0x00007fffb6380968 in io_getevents () from /lib64/libaio.so.1
#3 0x000000011568e1ec in LinuxAIOHandler::collect (this=0x7fffafffe440) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/os/os0file.cc:2326
#4 0x000000011568f9bc in LinuxAIOHandler::poll (this=0x7fffafffe440, m1=0x7fffafffe558, m2=0x7fffafffe560, request=0x7fffafffe568)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/os/os0file.cc:2471
#5 0x0000000115691ca8 in os_aio_linux_handler (request=0x7fffafffe568, m2=0x7fffafffe560, m1=0x7fffafffe558, global_segment=<optimized out>)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/os/os0file.cc:2520
#6 os_aio_handler (segment=<optimized out>, m1=0x7fffafffe558, m2=0x7fffafffe560, request=0x7fffafffe568)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/os/os0file.cc:5871
#7 0x0000000115a23c44 in fil_aio_wait (segment=0) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/fil/fil0fil.cc:7562
#8 0x00000001157be428 in io_handler_thread (segment=0) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/srv/srv0start.cc:267
#9 0x00000001157d25e8 in std::__invoke_impl<void, void (*&)(unsigned long), unsigned long&> (__f=<synthetic pointer>: <optimized out>) at /usr/include/c++/9/bits/invoke.h:89
#10 std::__invoke<void (*&)(unsigned long), unsigned long&> (__fn=<synthetic pointer>: <optimized out>) at /usr/include/c++/9/bits/invoke.h:95
#11 std::_Bind<void (*(unsigned long))(unsigned long)>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) (__args=..., this=<synthetic pointer>)
at /usr/include/c++/9/functional:400
#12 std::_Bind<void (*(unsigned long))(unsigned long)>::operator()<, void>() (this=<synthetic pointer>) at /usr/include/c++/9/functional:484
#13 Runnable::operator()<void (*)(unsigned long), unsigned long> (f=@0x7fffb01cb440: 0x1157be3e0 <io_handler_thread(ulint)>, this=0x7fffb01cb448)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/include/os0thread-create.h:91
#14 std::__invoke_impl<void, Runnable, void (*)(unsigned long), unsigned long> (__f=...) at /usr/include/c++/9/bits/invoke.h:60
#15 std::__invoke<Runnable, void (*)(unsigned long), unsigned long> (__fn=...) at /usr/include/c++/9/bits/invoke.h:95
#16 std::thread::_Invoker<std::tuple<Runnable, void (*)(unsigned long), unsigned long> >::_M_invoke<0ul, 1ul, 2ul> (this=0x7fffb01cb438) at /usr/include/c++/9/thread:244
#17 std::thread::_Invoker<std::tuple<Runnable, void (*)(unsigned long), unsigned long> >::operator() (this=0x7fffb01cb438) at /usr/include/c++/9/thread:251
#18 std::thread::_State_impl<std::thread::_Invoker<std::tuple<Runnable, void (*)(unsigned long), unsigned long> > >::_M_run (this=0x7fffb01cb430) at /usr/include/c++/9/thread:195
#19 0x00007fffb5ac3dd0 in ?? () from /lib64/libstdc++.so.6
#20 0x00007fffb83f8c10 in start_thread () from /lib64/libpthread.so.0
#21 0x00007fffb576d8a8 in clone () from /lib64/libc.so.6
(gdb) thread 13
[Switching to thread 13 (Thread 0x7fff9cf9f060 (LWP 11183))]
#0 0x00007fffb840186c in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
(gdb) where
#0 0x00007fffb840186c in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
#1 0x00000001156968b4 in os_event::wait (this=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/include/sync0types.h:536
#2 os_event::wait_low (this=0x7fffb0046918, reset_sig_count=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/os/os0event.cc:337
#3 0x0000000115696ef0 in os_event_wait_low (event=<optimized out>, reset_sig_count=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/os/os0event.cc:523
#4 0x000000011592e018 in buf_flush_page_coordinator_thread (n_page_cleaners=<optimized out>) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/buf/buf0flu.cc:3016
#5 0x00000001157d25e8 in std::__invoke_impl<void, void (*&)(unsigned long), unsigned long&> (__f=<synthetic pointer>: <optimized out>) at /usr/include/c++/9/bits/invoke.h:89
#6 std::__invoke<void (*&)(unsigned long), unsigned long&> (__fn=<synthetic pointer>: <optimized out>) at /usr/include/c++/9/bits/invoke.h:95
#7 std::_Bind<void (*(unsigned long))(unsigned long)>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) (__args=..., this=<synthetic pointer>)
at /usr/include/c++/9/functional:400
#8 std::_Bind<void (*(unsigned long))(unsigned long)>::operator()<, void>() (this=<synthetic pointer>) at /usr/include/c++/9/functional:484
#9 Runnable::operator()<void (*)(unsigned long), unsigned long> (f=@0x7fffb0214b00: 0x11592dda0 <buf_flush_page_coordinator_thread(size_t)>, this=0x7fffb0214b08)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/include/os0thread-create.h:91
#10 std::__invoke_impl<void, Runnable, void (*)(unsigned long), unsigned long> (__f=...) at /usr/include/c++/9/bits/invoke.h:60
#11 std::__invoke<Runnable, void (*)(unsigned long), unsigned long> (__fn=...) at /usr/include/c++/9/bits/invoke.h:95
#12 std::thread::_Invoker<std::tuple<Runnable, void (*)(unsigned long), unsigned long> >::_M_invoke<0ul, 1ul, 2ul> (this=0x7fffb0214af8) at /usr/include/c++/9/thread:244
#13 std::thread::_Invoker<std::tuple<Runnable, void (*)(unsigned long), unsigned long> >::operator() (this=0x7fffb0214af8) at /usr/include/c++/9/thread:251
#14 std::thread::_State_impl<std::thread::_Invoker<std::tuple<Runnable, void (*)(unsigned long), unsigned long> > >::_M_run (this=0x7fffb0214af0) at /usr/include/c++/9/thread:195
#15 0x00007fffb5ac3dd0 in ?? () from /lib64/libstdc++.so.6
#16 0x00007fffb83f8c10 in start_thread () from /lib64/libpthread.so.0
#17 0x00007fffb576d8a8 in clone () from /lib64/libc.so.6
(gdb) thread 14
[Switching to thread 14 (Thread 0x7fff797cf060 (LWP 11190))]
#0 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
(gdb) where
#0 0x00007fffb8401f34 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib64/libpthread.so.0
#1 0x0000000115697164 in os_event::timed_wait (this=<error reading variable: value has been optimized out>, abstime=<optimized out>)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/include/sync0types.h:536
#2 0x0000000115697438 in os_event::wait_time_low (this=0x7fffb0215c38, time_in_usec=<optimized out>, reset_sig_count=<optimized out>)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/os/os0event.cc:410
#3 0x0000000115697600 in os_event_wait_time_low (event=<optimized out>, time_in_usec=<optimized out>, reset_sig_count=<optimized out>)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/os/os0event.cc:509
#4 0x000000011563ea88 in log_checkpointer (log_ptr=0x7fffb0218e00) at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/log/log0chkp.cc:924
#5 0x0000000115653138 in std::__invoke_impl<void, void (*&)(log_t*), log_t*&> (__f=<synthetic pointer>: <optimized out>) at /usr/include/c++/9/bits/invoke.h:89
#6 std::__invoke<void (*&)(log_t*), log_t*&> (__fn=<synthetic pointer>: <optimized out>) at /usr/include/c++/9/bits/invoke.h:95
#7 std::_Bind<void (*(log_t*))(log_t*)>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) (__args=..., this=<synthetic pointer>) at /usr/include/c++/9/functional:400
#8 std::_Bind<void (*(log_t*))(log_t*)>::operator()<, void>() (this=<synthetic pointer>) at /usr/include/c++/9/functional:484
#9 Runnable::operator()<void (*)(log_t*), log_t*> (f=@0x7fffb030c300: 0x11563e5e0 <log_checkpointer(log_t*)>, this=0x7fffb030c308)
at /var/lib/mysql/rpmbuild/BUILD/mysql-8.0.16/storage/innobase/include/os0thread-create.h:91
#10 std::__invoke_impl<void, Runnable, void (*)(log_t*), log_t*> (__f=...) at /usr/include/c++/9/bits/invoke.h:60
#11 std::__invoke<Runnable, void (*)(log_t*), log_t*> (__fn=...) at /usr/include/c++/9/bits/invoke.h:95
#12 std::thread::_Invoker<std::tuple<Runnable, void (*)(log_t*), log_t*> >::_M_invoke<0ul, 1ul, 2ul> (this=0x7fffb030c2f8) at /usr/include/c++/9/thread:244
#13 std::thread::_Invoker<std::tuple<Runnable, void (*)(log_t*), log_t*> >::operator() (this=0x7fffb030c2f8) at /usr/include/c++/9/thread:251
#14 std::thread::_State_impl<std::thread::_Invoker<std::tuple<Runnable, void (*)(log_t*), log_t*> > >::_M_run (this=0x7fffb030c2f0) at /usr/include/c++/9/thread:195
#15 0x00007fffb5ac3dd0 in ?? () from /lib64/libstdc++.so.6
#16 0x00007fffb83f8c10 in start_thread () from /lib64/libpthread.so.0
#17 0x00007fffb576d8a8 in clone () from /lib64/libc.so.6
(gdb)