MySQL Forums
Forum List  »  InnoDB

MySQL 8.013 server in startup crash loop
Posted by: C D Tavares
Date: January 23, 2019 08:55PM

I am not a server administrator, just a schlub who installed MySQL some time back to keep a simple inventory database.

I upgraded around January 10 to this version from a much older version (5?) which I don't believe made much use of the innoDB engine... but the upgrade changed everything to innoDB as some sort of default (I certainly didn't ask for it).

Today, my client app was unable to connect. I found that the server has been crashing and restarting continually every 15 seconds or so for a little over 24 hours. I have 6645 binlog.* files in data, and a huge mysqld.local.err file that consists of many repetitions of the same boot crash data (one iteration of which is appended here at the bottom).

Significant message is:
[ERROR] [MY-013183] [InnoDB] Assertion failure: dict0dict.cc:3289:for_table || ref_table thread

Among the advice in the dump is:

InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.

Going there, I found advice to:

...add the following line to the [mysqld] section of your option file before restarting the server:
[mysqld]
innodb_force_recovery = 1

Further investigation led me to "mysqld --verbose --help", which informed me:

Default options are read from the following files in the given order:
/etc/my.cnf /etc/mysql/my.cnf /usr/local/mysql/etc/my.cnf ~/.my.cnf

The problem is that NONE OF THOSE FILES EXIST. My backup media from a month ago indicate that they did not exist in the previous version I was running, either.

This is a standard macOS installation from the DMG installer package, on Sierra. Is this configuration file supposedly required? I would create one just to be able to run the innoDB recovery stuff, but I'm sure it has plenty of other required syntax and I don't have a base instance to start from.

Here is one iteration of the dump information:

2019-01-24T01:09:25.939195Z 0 [System] [MY-010116] [Server] /usr/local/mysql/bin/mysqld (mysqld 8.0.13) starting as process 1135
2019-01-24T01:09:25.942479Z 0 [Warning] [MY-010159] [Server] Setting lower_case_table_names=2 because file system for /usr/local/mysql/data/ is case insensitive
2019-01-24T01:09:26.864913Z 0 [System] [MY-010229] [Server] Starting crash recovery...
2019-01-24T01:09:26.877074Z 0 [System] [MY-010232] [Server] Crash recovery finished.
2019-01-24T01:09:26.894635Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2019-01-24T01:09:26.913192Z 0 [System] [MY-010931] [Server] /usr/local/mysql/bin/mysqld: ready for connections. Version: '8.0.13' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Server - GPL.
2019-01-24T01:09:27.112083Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: '/tmp/mysqlx.sock' bind-address: '::' port: 33060
2019-01-24T01:09:27.888947Z 0 [ERROR] [MY-013183] [InnoDB] Assertion failure: dict0dict.cc:3289:for_table || ref_table thread 123145473007616
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
01:09:27 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.

key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=1
max_threads=151
thread_count=1
connection_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 67867 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7f9cd60f2c00
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 70000a2c9df0 thread_stack 0x46000
0 mysqld 0x0000000103c8ad4c my_print_stacktrace(unsigned char*, unsigned long) + 60
1 mysqld 0x00000001030b8456 handle_fatal_signal + 678
2 libsystem_platform.dylib 0x00007fffbe3ceb3a _sigtramp + 26
3 ??? 0x0000000100000001 0x0 + 4294967297
4 libsystem_c.dylib 0x00007fffbe253420 abort + 129
5 mysqld 0x0000000104033e99 ut_dbg_assertion_failed(char const*, char const*, unsigned long) + 329
6 mysqld 0x0000000103dd211b dict_foreign_add_to_cache(dict_foreign_t*, char const**, bool, bool, dict_err_ignore_t) + 1707
7 mysqld 0x0000000103deb867 dd_table_load_fk_from_dd(dict_table_t*, dd::Table const*, char const**, dict_err_ignore_t, bool) + 1543
8 mysqld 0x0000000103df1aa3 dict_table_t* dd_open_table_one<dd::Table>(dd::cache::Dictionary_client*, TABLE const*, char const*, dd::Table const*, THD*, std::__1::deque<char const*, ut_allocator<char const*> >&) + 8403
9 mysqld 0x0000000103de2315 dd_table_open_on_dd_obj(dd::cache::Dictionary_client*, dd::Table const&, dd::Partition const*, char const*, dict_table_t*&, THD*) + 2213
10 mysqld 0x0000000103de3595 dd_table_open_on_id_low(THD*, MDL_ticket**, unsigned long long) + 1717
11 mysqld 0x0000000103de2d1b dd_table_open_on_id(unsigned long long, THD*, MDL_ticket**, bool, bool) + 2043
12 mysqld 0x0000000103fc4a71 row_purge_step(que_thr_t*) + 753
13 mysqld 0x0000000103f877f7 que_run_threads(que_thr_t*) + 679
14 mysqld 0x000000010400f188 trx_purge(unsigned long, unsigned long, bool) + 3720
15 mysqld 0x0000000103ff0207 srv_purge_coordinator_thread() + 2359
16 mysqld 0x0000000103ec7ce6 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, Runnable, void (*)()> >(void*) + 118
17 libsystem_pthread.dylib 0x00007fffbe3d893b _pthread_body + 180
18 libsystem_pthread.dylib 0x00007fffbe3d8887 _pthread_body + 0
19 libsystem_pthread.dylib 0x00007fffbe3d808d thread_start + 13

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0): is an invalid pointer
Connection ID (thread ID): 0
Status: NOT_KILLED

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

Options: ReplyQuote


Subject
Views
Written By
Posted
MySQL 8.013 server in startup crash loop
102
January 23, 2019 08:55PM


Sorry, only registered users may post in this forum.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.