Re: Connection "delayed" attempts
Posted by: Rick James
Date: May 06, 2016 12:23PM

While you may have found one issue, I spotted several others:

**Observations:**
Version: 5.7.9-log
32 GB of RAM
Uptime = 2d 22:02:37
You are not running on Windows.
Running 64-bit version
You appear to be running entirely (or mostly) MyISAM.


**The More Important Issues**

Migrate the tables from MyISAM to InnoDB.
See http://mysql.rjweb.org/doc.php/myisam2innodb

binlog_cache_size is dangerously high, especially considering the number of connections you get to.
Leave it at 256K, not 256M.

Since you are mostly using MyISAM, change the caching to
key_buffer_size = 2G
innodb_buffer_pool_size = 3G
(It would probably be beneficial to convert to InnoDB, but that is another discussion.)

Decrease tmp_table_size and max_heap_table_size to, say, 100M.

There are lots of tables and/or lots of opening of tables. What's going on? 902 tables opened per second!

There are a lots of poorly performing queries. Set long_query_time = 1 and turn on the slowlog to find out which ones they are. And let's discuss how to improve them. (Increasing tmp_table_size is _not_ the answer.)

Why is SHOW VARIABLES happening several times per second?! And 31 USE statements per second? Is this some sloppy 3rd party software?

There are 19 connections per second. Make sure this is not coming from multiple connections within a single web page.

Increase thread_cache_size to 50.

Decrease max_allowed_packet to 150M unless you have some particular reason for the high value.

The QUery cache is not being very effective.
Plan A: Increase query_cache_size to 60M, but no more.
Plan B: Change query_cache_type to DEMAND and pepper the SELECTs with SQL_CACHE and SQL_NO_CACHE as appropriate.
Plan C: Turn off the Query cache -- type=OFF and size=0.

Increase binlog_stmt_cache_size to 64K. Then monitor Binlog_stmt_cache_disk_use / Uptime. Get it below 1/sec.

There are an awful lot of SHOW commands being executed; what is going on? For example, why would anyone need 4.3 SHOW COLLATIONs per second?


**Details and other observations**

Memory allocation : http://mysql.rjweb.org/doc.php/memory

( innodb_buffer_pool_size / _ram ) = 27,917,287,424 / 32768M = 81.2% -- % of RAM used for InnoDB buffer_pool
Too high, especially since you are not using InnoDB that much.

( table_open_cache ) = 10,000 -- Number of table descriptors to cache
-- Several hundred is usually good.

( innodb_change_buffer_max_size ) = 50 -- Percent of buffer_pool that is used for "change buffer" -- a write cache for index changes.
50% seems excessively large.

( min( tmp_table_size, max_heap_table_size ) / _ram ) = min( 1932M, 1932M ) / 32768M = 5.9% -- Percent of RAM to allocate when needing MEMORY table (per table), or temp table inside a SELECT (per temp table per some SELECTs). Too high may lead to swapping.
-- Decrease tmp_table_size and max_heap_table_size to, say, 1% of ram.

( myisam_sort_buffer_size / _ram ) = 1024M / 32768M = 3.1% -- Used for ALTER, CREATE INDEX, OPTIMIZE, LOAD DATA; set when you need it. Also for MyISAM's REPAIR TABLE.
-- Decrease myisam_sort_buffer_size to keep from blowing out RAM.

( Queries ) = 1,338,882,490 / 252157 = 5309 /sec -- Queries (including inside SP)
-- >3000 _may_ be stressing server

( Created_tmp_tables ) = 106,807,225 / 252157 = 423 /sec -- Frequency of creating "temp" tables as part of complex SELECTs.

( Created_tmp_disk_tables ) = 17,885,626 / 252157 = 71 /sec -- Frequency of creating _disk_ "temp" tables as part of complex SELECTs
-- increase tmp_table_size and max_heap_table_size.
Check the rules for temp tables on when MEMORY is used instead of MyISAM. Perhaps minor schema or query changes can avoid MyISAM.
Better indexes and reformulation of queries are more likely to help.

( tmp_table_size ) = 1932M -- Limit on size of _MEMORY_ temp tables used to support a SELECT
-- Decrease tmp_table_size to avoid running out of RAM. Perhaps no more than 64M.

( Com_show_variables ) = 1,147,878 / 252157 = 4.6 /sec -- SHOW VARIABLES ...
-- Why are you requesting the VARIABLES so often?

( Select_scan ) = 5,118,282 / 252157 = 20 /sec -- full table scans
-- Add indexes / optimize queries (unless they are tiny tables)

( Com_insert + Com_delete + Com_delete_multi + Com_replace + Com_update + Com_update_multi ) = (42969061 + 5200869 + 4203 + 23068 + 56512809 + 79125) / 252157 = 415 /sec -- writes/sec
-- 50 writes/sec + log flushes will probably max out I/O write capacity of normal drives

( ( Com_stmt_prepare - Com_stmt_close ) / ( Com_stmt_prepare + Com_stmt_close ) ) = ( 3472408 - 2383130 ) / ( 3472408 + 2383130 ) = 18.6% -- Are you closing your prepared statements?
-- Add Closes.

( expire_logs_days ) = 1 -- How soon to automatically purge binlog (after this many days)
-- Too large (or zero) = consumes disk space; too small = need to respond quickly to network/machine crash.

( long_query_time ) = 10.000000 = 10 -- Cutoff (Seconds) for defining a "slow" query.
-- Suggest 2

( Slow_queries ) = 771,003 / 252157 = 3.1 /sec -- Frequency (Slow queries per sec)
-- Rework slow guys; improve indexes; watch disk space for slow log file

( back_log / max_connections ) = 3,000 / 1000 = 300.0%

( Com_change_db ) = 7,806,156 / 252157 = 31 /sec -- Probably comes from USE statements.
-- Consider connecting with DB, using db.tbl syntax, eliminating spurious USE statements, etc.

( max_connect_errors ) = 1,000,000 = 1.0e+6 -- A small protection against hackers.
-- Perhaps no more than 200.

( Connections ) = 4,697,155 / 252157 = 19 /sec -- Connections
-- Increase wait_timeout; use pooling?

( Threads_created / Connections ) = 66,604 / 4697155 = 1.4% -- Rapidity of process creation
-- Increase thread_cache_size (non-Windows)

max_allowed_packet = 1G -- This is rather high. Recommend lowering it unless you have some argument for it. slave_pending_jobs_size_max would also need shrinking.

Aborted_connections = 11/minute -- Why so high?

Binlog_stmt_cache_disk_use = 1.5/sec -- Very high. Are there a lot of 'big' writes?

5639 Open_tables -- why so many?

log_queries_not_using_indexes is ON -- I prefer OFF because of all the clutter in the slowlog.

slave_skip_errors -- Setting this is "sweeping problems under the rug".

Options: ReplyQuote


Subject
Written By
Posted
Re: Connection "delayed" attempts
May 06, 2016 12:23PM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.