have to restart mysql every day

[Andrew Mel]

Hi,

after a lot of (mostly hardware related) problems setting up my first server (debian on a virtual host), he is now running since some month. One month ago I had to restart mysql after I got no emails anymore. dovecot/postfix wrote that they can't connect to mysql. I restarted it and it worked again. But I have to restart it very often (once a day?!). Today I saw in the syslog, that it was also automatically restarted by mysqld_safe (when I was not fast enough to do it manually?!). For example yesterday I restarted it at 15:45. At 16:38 mysql was not reachable again. At 17:31 its automatically restarted

Mar 15 16:38:43 masta dovecot: auth-worker(24764): Error: mysql(localhost): Connect failed to database (dbispconfig): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) - waiting for 1 seconds before retry

...

Mar 15 17:31:32 masta mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 [Note] Plugin 'FEDERATED' is disabled.
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 InnoDB: The InnoDB memory heap is disabled
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 InnoDB: Mutexes and rw_locks use GCC atomic builtins
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 InnoDB: Compressed tables use zlib 1.2.7
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 InnoDB: Using Linux native AIO
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 InnoDB: Initializing buffer pool, size = 128.0M
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 InnoDB: Completed initialization of buffer pool
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 InnoDB: highest supported file format is Barracuda.
Mar 15 17:31:32 masta mysqld: InnoDB: The log sequence number in ibdata files does not match
Mar 15 17:31:32 masta mysqld: InnoDB: the log sequence number in the ib_logfiles!
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 InnoDB: Database was not shut down normally!
Mar 15 17:31:32 masta mysqld: InnoDB: Starting crash recovery.
Mar 15 17:31:32 masta mysqld: InnoDB: Reading tablespace information from the .ibd files...
Mar 15 17:31:32 masta mysqld: InnoDB: Restoring possible half-written data pages from the doublewrite
Mar 15 17:31:32 masta mysqld: InnoDB: buffer...
Mar 15 17:31:32 masta mysqld: 140315 17:31:32 InnoDB: Waiting for the background threads to start
Mar 15 17:31:33 masta mysqld: 140315 17:31:33 InnoDB: 5.5.33 started; log sequence number 62788739
Mar 15 17:31:33 masta mysqld: 140315 17:31:33 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
Mar 15 17:31:33 masta mysqld: 140315 17:31:33 [Note] - '127.0.0.1' resolves to '127.0.0.1';
Mar 15 17:31:33 masta mysqld: 140315 17:31:33 [Note] Server socket created on IP: '127.0.0.1'.
Mar 15 17:31:33 masta mysqld: 140315 17:31:33 [Note] Event Scheduler: Loaded 0 events
Mar 15 17:31:33 masta mysqld: 140315 17:31:33 [Note] /usr/sbin/mysqld: ready for connections.
Mar 15 17:31:33 masta mysqld: Version: '5.5.33-0+wheezy1' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Debian)
Mar 15 17:31:34 masta /etc/mysql/debian-start[30830]: Upgrading MySQL tables if necessary.
Mar 15 17:31:34 masta /etc/mysql/debian-start[30835]: /usr/bin/mysql_upgrade: the '--basedir' option is always ignored
Mar 15 17:31:34 masta /etc/mysql/debian-start[30835]: Looking for 'mysql' as: /usr/bin/mysql
Mar 15 17:31:34 masta /etc/mysql/debian-start[30835]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
Mar 15 17:31:34 masta /etc/mysql/debian-start[30835]: This installation of MySQL is already upgraded to 5.5.33, use --force if you still need to run mysql_upgrade
Mar 15 17:31:34 masta /etc/mysql/debian-start[30851]: Checking for insecure root accounts.
Mar 15 17:31:34 masta /etc/mysql/debian-start[30857]: Triggering myisam-recover for all MyISAM tables
Mar 15 17:31:34 masta mysqld: 140315 17:31:34 [ERROR] /usr/sbin/mysqld: Table './dbispconfig/mail_traffic' is marked as crashed and should be repaired
Mar 15 17:31:34 masta mysqld: 140315 17:31:34 [Warning] Checking table: './dbispconfig/mail_traffic'
Mar 15 17:31:34 masta mysqld: 140315 17:31:34 [ERROR] /usr/sbin/mysqld: Table './dbispconfig/monitor_data' is marked as crashed and should be repaired
Mar 15 17:31:34 masta mysqld: 140315 17:31:34 [Warning] Checking table: './dbispconfig/monitor_data'

"Should be repaired" doesn't sound very nice. Do I have to repair it manually?


Today there are some even more creepy lines in the syslog

Mar 16 16:22:35 masta mysqld: InnoDB: Starting crash recovery.
Mar 16 16:22:35 masta mysqld: InnoDB: Reading tablespace information from the .ibd files...
Mar 16 16:22:35 masta mysqld: InnoDB: Restoring possible half-written data pages from the doublewrite
Mar 16 16:22:35 masta mysqld: InnoDB: buffer...
Mar 16 16:22:35 masta mysqld: InnoDB: Error: tried to read 16384 bytes at offset 0 7995392.
Mar 16 16:22:35 masta mysqld: InnoDB: Was only able to read 8192.
Mar 16 16:22:35 masta mysqld: InnoDB: Fatal error: cannot read from file. OS error number 17.
Mar 16 16:22:35 masta mysqld: 140316 16:22:35 InnoDB: Assertion failure in thread 4138494176 in file os0file.c line 2538
Mar 16 16:22:35 masta mysqld: InnoDB: We intentionally generate a memory trap.
Mar 16 16:22:35 masta mysqld: InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
Mar 16 16:22:35 masta mysqld: InnoDB: If you get repeated assertion failures or crashes, even
Mar 16 16:22:35 masta mysqld: InnoDB: immediately after the mysqld startup, there may be
Mar 16 16:22:35 masta mysqld: InnoDB: corruption in the InnoDB tablespace. Please refer to
Mar 16 16:22:35 masta mysqld: InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
Mar 16 16:22:35 masta mysqld: InnoDB: about forcing recovery.
Mar 16 16:22:35 masta mysqld: 15:22:35 UTC - mysqld got signal 6 ;
Mar 16 16:22:35 masta mysqld: This could be because you hit a bug. It is also possible that this binary
Mar 16 16:22:35 masta mysqld: or one of the libraries it was linked against is corrupt, improperly built,
Mar 16 16:22:35 masta mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
Mar 16 16:22:35 masta mysqld: We will try our best to scrape up some info that will hopefully help
Mar 16 16:22:35 masta mysqld: diagnose the problem, but since we have already crashed,
Mar 16 16:22:35 masta mysqld: something is definitely wrong and this may fail.

Followed by lots of lines with values of variables

I tried to add skip-innodb to the my.cnf but mysql wont restart after that.

I think the only thing I changed in the cnf was the line innodb_use_native_aio = 0 after the following error in january:

Jan 3 16:26:10 masta mysqld: 140103 16:26:10 InnoDB: Error: io_setup() failed with EAGAIN after 5 attempts.
Jan 3 16:26:10 masta mysqld: InnoDB: You can disable Linux Native AIO by setting innodb_use_native_aio = 0 in my.cnf

Does anybody have any other advice what I can try to find more data why mysql is not reachable. The mysql log is empty.

ok....update...
general_log_file = /var/log/mysql/mysql.log was commented out in the my.cnf...I have changed that now.

Any other advice what I can do?
Thanks!