MySQL Forums
Forum List  »  Performance

Optimizing the MySQL server performance
Posted by: bob ly
Date: September 21, 2005 04:29PM

Hi,

I have a Dell SC1420 poweredge with RHEL ES3 and 2Gb ram. The server performance is really sloppy at times but really fast few other times. I have perl scripts which are running every half hour and loads the data. When the scripst are running in the background and if someone access through the web , the performance is really bad. It may take around 1.5-2 minutes to fetch records. But sometimes it takes only < 10seconds. I am using insert delayed in my perl scripts while the data is being loaded so that read has more priority. This hasn't helped either. I have tried every bit to optimize it, but nothing has worked so far. Would really appreciate if someone could help me to get the max performance. I also changed the table_cache to 512 from 256 but this si not getting reflected in show status.

Here is the my.cnf file
*********************************************************
# The MySQL server
[mysqld]
port = 3306
socket = /var/lib/mysql/mysql.sock
datadir = /space/var/lib/mysql
skip-locking
key_buffer = 756M
max_allowed_packet = 1M
table_cache = 512
sort_buffer_size = 4M
read_buffer_size = 2M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache = 8
query_cache_size= 32M
skip-innodb
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 4

# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (via the "enable-named-pipe" option) will render mysqld useless!
#
#skip-networking

# Replication Master Server (default)
# binary logging is required for replication
#log-bin

# required unique id between 1 and 2^32 - 1
# defaults to 1 if master-host is not set
# but will not function as a master if omitted
server-id = 1

# Replication Slave (comment out master section to use this)
#
# To configure this host as a replication slave, you can choose between
# two methods :
#
# 1) Use the CHANGE MASTER TO command (fully described in our manual) -
# the syntax is:
#
# CHANGE MASTER TO MASTER_HOST=<host>, MASTER_PORT=<port>,
# MASTER_USER=<user>, MASTER_PASSWORD=<password> ;
#
# where you replace <host>, <user>, <password> by quoted strings and
# <port> by the master's port number (3306 by default).
#
# Example:
#
# CHANGE MASTER TO MASTER_HOST='125.564.12.1', MASTER_PORT=3306,
# MASTER_USER='joe', MASTER_PASSWORD='secret';
#
# OR
#
# 2) Set the variables below. However, in case you choose this method, then
# start replication for the first time (even unsuccessfully, for example
# if you mistyped the password in master-password and the slave fails to
# connect), the slave will create a master.info file, and any later
# change in this file to the variables' values below will be ignored and
# overridden by the content of the master.info file, unless you shutdown
# the slave server, delete master.info and restart the slaver server.
# For that reason, you may want to leave the lines below untouched
# (commented) and instead use CHANGE MASTER TO (see above)
#
# required unique id between 2 and 2^32 - 1
# (and different from the master)
# defaults to 2 if master-host is set
# but will not function as a slave if omitted
#server-id = 2
#
# The replication master for this slave - required
#master-host = <hostname>
#
# The username the slave will use for authentication when connecting
# to the master - required
#master-user = <username>
#
# The password the slave will authenticate with when connecting to
# the master - required
#master-password = <password>
#
# The port the master is listening on.
# optional - defaults to 3306
#master-port = <port>
#
# binary logging - not required for slaves, but recommended
#log-bin

# Point the following paths to different dedicated disks
#tmpdir = /tmp/
#log-update = /path-to-dedicated-directory/hostname

# Uncomment the following if you are using BDB tables
#bdb_cache_size = 64M
#bdb_max_lock = 100000

# Uncomment the following if you are using InnoDB tables
#innodb_data_home_dir = /var/lib/mysql/
#innodb_data_file_path = ibdata1:10M:autoextend
#innodb_log_group_home_dir = /var/lib/mysql/
#innodb_log_arch_dir = /var/lib/mysql/
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
#innodb_buffer_pool_size = 256M
#innodb_additional_mem_pool_size = 20M
# Set .._log_file_size to 25 % of buffer pool size
#innodb_log_file_size = 64M
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 50

[safe_mysqld]
err-log=/space/var/log/mysqld.log
pid-file=/var/lib/mysql/localhost.localdomain.pid

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

**********************************************************
here is my variables;
show variables;
+---------------------------------+------------------------------------------------+
| Variable_name | Value |
+---------------------------------+------------------------------------------------+
| back_log | 50 |
| basedir | / |
| binlog_cache_size | 32768 |
| bulk_insert_buffer_size | 8388608 |
| character_set_client | latin1 |
| character_set_connection | latin1 |
| character_set_database | latin1 |
| character_set_results | latin1 |
| character_set_server | latin1 |
| character_set_system | utf8 |
| character_sets_dir | /usr/share/mysql/charsets/ |
| collation_connection | latin1_swedish_ci |
| collation_database | latin1_swedish_ci |
| collation_server | latin1_swedish_ci |
| concurrent_insert | ON |
| connect_timeout | 5 |
| datadir | /space/var/lib/mysql/ |
| date_format | %Y-%m-%d |
| datetime_format | %Y-%m-%d %H:%i:%s |
| default_week_format | 0 |
| delay_key_write | ON |
| delayed_insert_limit | 100 |
| delayed_insert_timeout | 300 |
| delayed_queue_size | 1000 |
| expire_logs_days | 0 |
| flush | OFF |
| flush_time | 0 |
| ft_boolean_syntax | + -><()~*:""&| |
| ft_max_word_len | 84 |
| ft_min_word_len | 4 |
| ft_query_expansion_limit | 20 |
| ft_stopword_file | (built-in) |
| group_concat_max_len | 1024 |
| have_archive | NO |
| have_bdb | NO |
| have_blackhole_engine | NO |
| have_compress | YES |
| have_crypt | YES |
| have_csv | NO |
| have_example_engine | NO |
| have_geometry | YES |
| have_innodb | DISABLED |
| have_isam | NO |
| have_ndbcluster | NO |
| have_openssl | NO |
| have_query_cache | YES |
| have_raid | NO |
| have_rtree_keys | YES |
| have_symlink | YES |
| init_connect | |
| init_file | |
| init_slave | |
| innodb_additional_mem_pool_size | 1048576 |
| innodb_autoextend_increment | 8 |
| innodb_buffer_pool_awe_mem_mb | 0 |
| innodb_buffer_pool_size | 8388608 |
| innodb_data_file_path | |
| innodb_data_home_dir | |
| innodb_fast_shutdown | ON |
| innodb_file_io_threads | 4 |
| innodb_file_per_table | OFF |
| innodb_flush_log_at_trx_commit | 1 |
| innodb_flush_method | |
| innodb_force_recovery | 0 |
| innodb_lock_wait_timeout | 50 |
| innodb_locks_unsafe_for_binlog | OFF |
| innodb_log_arch_dir | |
| innodb_log_archive | OFF |
| innodb_log_buffer_size | 1048576 |
| innodb_log_file_size | 5242880 |
| innodb_log_files_in_group | 2 |
| innodb_log_group_home_dir | |
| innodb_max_dirty_pages_pct | 90 |
| innodb_max_purge_lag | 0 |
| innodb_mirrored_log_groups | 1 |
| innodb_open_files | 300 |
| innodb_table_locks | ON |
| innodb_thread_concurrency | 8 |
| interactive_timeout | 28800 |
| join_buffer_size | 131072 |
| key_buffer_size | 792723456 |
| key_cache_age_threshold | 300 |
| key_cache_block_size | 1024 |
| key_cache_division_limit | 100 |
| language | /usr/share/mysql/english/ |
| large_files_support | ON |
| license | GPL |
| local_infile | ON |
| locked_in_memory | OFF |
| log | OFF |
| log_bin | OFF |
| log_error | |
| log_slave_updates | OFF |
| log_slow_queries | OFF |
| log_update | OFF |
| log_warnings | 1 |
| long_query_time | 10 |
| low_priority_updates | ON |
| lower_case_file_system | OFF |
| lower_case_table_names | 0 |
| max_allowed_packet | 1047552 |
| max_binlog_cache_size | 4294967295 |
| max_binlog_size | 1073741824 |
| max_connect_errors | 10 |
| max_connections | 100 |
| max_delayed_threads | 20 |
| max_error_count | 64 |
| max_heap_table_size | 16777216 |
| max_insert_delayed_threads | 20 |
| max_join_size | 4294967295 |
| max_length_for_sort_data | 1024 |
| max_relay_log_size | 0 |
| max_seeks_for_key | 4294967295 |
| max_sort_length | 1024 |
| max_tmp_tables | 32 |
| max_user_connections | 0 |
| max_write_lock_count | 4294967295 |
| myisam_data_pointer_size | 4 |
| myisam_max_extra_sort_file_size | 2147483648 |
| myisam_max_sort_file_size | 2147483647 |
| myisam_recover_options | OFF |
| myisam_repair_threads | 1 |
| myisam_sort_buffer_size | 67108864 |
| net_buffer_length | 16384 |
| net_read_timeout | 30 |
| net_retry_count | 10 |
| net_write_timeout | 60 |
| new | OFF |
| old_passwords | ON |
| open_files_limit | 1024 |
| pid_file | /space/var/lib/mysql/localhost.localdomain.pid |
| port | 3306 |
| preload_buffer_size | 32768 |
| protocol_version | 10 |
| query_alloc_block_size | 8192 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 16777216 |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
| query_prealloc_size | 8192 |
| range_alloc_block_size | 2048 |
| read_buffer_size | 2097152 |
| read_only | OFF |
| read_rnd_buffer_size | 4190208 |
| relay_log_purge | ON |
| relay_log_space_limit | 0 |
| rpl_recovery_rank | 0 |
| secure_auth | OFF |
| server_id | 1 |
| skip_external_locking | ON |
| skip_networking | OFF |
| skip_show_database | OFF |
| slave_net_timeout | 3600 |
| slave_transaction_retries | 0 |
| slow_launch_time | 2 |
| socket | /var/lib/mysql/mysql.sock |
| sort_buffer_size | 4194304 |
| sql_mode | |
| storage_engine | MyISAM |
| sql_notes | OFF |
| sql_warnings | OFF |
| sync_binlog | 0 |
| sync_replication | 0 |
| sync_replication_slave_id | 0 |
| sync_replication_timeout | 0 |
| sync_frm | ON |
| system_time_zone | EDT |
| table_cache | 256 |
| table_type | MyISAM |
| thread_cache_size | 40 |
| thread_stack | 126976 |
| time_format | %H:%i:%s |
| time_zone | SYSTEM |
| tmp_table_size | 33554432 |
| tmpdir | |
| transaction_alloc_block_size | 8192 |
| transaction_prealloc_size | 4096 |
| tx_isolation | REPEATABLE-READ |
| version | 4.1.11-standard |
| version_comment | MySQL Community Edition - Standard (GPL) |
| version_compile_machine | i686 |
| version_compile_os | pc-linux-gnu |
| wait_timeout | 28800 |
+---------------------------------+------------------------------------------------+
*************************************************

show status;
+----------------------------+------------+
| Variable_name | Value |
+----------------------------+------------+
| Aborted_clients | 14 |
| Aborted_connects | 2 |
| Binlog_cache_disk_use | 0 |
| Binlog_cache_use | 0 |
| Bytes_received | 2278145998 |
| Bytes_sent | 944033544 |
| Com_admin_commands | 44 |
| Com_alter_db | 0 |
| Com_alter_table | 262 |
| Com_analyze | 0 |
| Com_backup_table | 0 |
| Com_begin | 0 |
| Com_change_db | 12434 |
| Com_change_master | 0 |
| Com_check | 0 |
| Com_checksum | 0 |
| Com_commit | 0 |
| Com_create_db | 1 |
| Com_create_function | 0 |
| Com_create_index | 0 |
| Com_create_table | 370 |
| Com_dealloc_sql | 0 |
| Com_delete | 568 |
| Com_delete_multi | 0 |
| Com_do | 0 |
| Com_drop_db | 1 |
| Com_drop_function | 0 |
| Com_drop_index | 0 |
| Com_drop_table | 255 |
| Com_drop_user | 0 |
| Com_execute_sql | 0 |
| Com_flush | 0 |
| Com_grant | 0 |
| Com_ha_close | 0 |
| Com_ha_open | 0 |
| Com_ha_read | 0 |
| Com_help | 0 |
| Com_insert | 8613741 |
| Com_insert_select | 9941 |
| Com_kill | 1 |
| Com_load | 12 |
| Com_load_master_data | 0 |
| Com_load_master_table | 0 |
| Com_lock_tables | 0 |
| Com_optimize | 279 |
| Com_preload_keys | 0 |
| Com_prepare_sql | 0 |
| Com_purge | 0 |
| Com_purge_before_date | 0 |
| Com_rename_table | 0 |
| Com_repair | 0 |
| Com_replace | 0 |
| Com_replace_select | 56 |
| Com_reset | 0 |
| Com_restore_table | 0 |
| Com_revoke | 0 |
| Com_revoke_all | 0 |
| Com_rollback | 0 |
| Com_savepoint | 0 |
| Com_select | 63644 |
| Com_set_option | 324 |
| Com_show_binlog_events | 0 |
| Com_show_binlogs | 0 |
| Com_show_charsets | 8 |
| Com_show_collations | 34 |
| Com_show_column_types | 0 |
| Com_show_create_db | 4 |
| Com_show_create_table | 38 |
| Com_show_databases | 33 |
| Com_show_errors | 0 |
| Com_show_fields | 1440 |
| Com_show_grants | 0 |
| Com_show_innodb_status | 14048 |
| Com_show_keys | 313 |
| Com_show_logs | 0 |
| Com_show_master_status | 2 |
| Com_show_new_master | 0 |
| Com_show_open_tables | 0 |
| Com_show_privileges | 0 |
| Com_show_processlist | 353 |
| Com_show_slave_hosts | 0 |
| Com_show_slave_status | 0 |
| Com_show_status | 14051 |
| Com_show_storage_engines | 0 |
| Com_show_tables | 73 |
| Com_show_variables | 16 |
| Com_show_warnings | 0 |
| Com_slave_start | 0 |
| Com_slave_stop | 0 |
| Com_truncate | 67 |
| Com_unlock_tables | 0 |
| Com_update | 181 |
| Com_update_multi | 9 |
| Connections | 9725 |
| Created_tmp_disk_tables | 5 |
| Created_tmp_files | 11 |
| Created_tmp_tables | 67833 |
| Delayed_errors | 0 |
| Delayed_insert_threads | 0 |
| Delayed_writes | 941994 |
| Flush_commands | 4 |
| Handler_commit | 0 |
| Handler_delete | 23506762 |
| Handler_discover | 0 |
| Handler_read_first | 145 |
| Handler_read_key | 312304230 |
| Handler_read_next | 351235763 |
| Handler_read_prev | 68997 |
| Handler_read_rnd | 12228971 |
| Handler_read_rnd_next | 1330438166 |
| Handler_rollback | 0 |
| Handler_update | 247216357 |
| Handler_write | 164770298 |
| Key_blocks_not_flushed | 3 |
| Key_blocks_unused | 136248 |
| Key_blocks_used | 686838 |
| Key_read_requests | 652560751 |
| Key_reads | 5940861 |
| Key_write_requests | 139698242 |
| Key_writes | 27725501 |
| Max_used_connections | 16 |
| Not_flushed_delayed_rows | 0 |
| Open_files | 457 |
| Open_streams | 0 |
| Open_tables | 250 |
| Opened_tables | 2121 |
| Qcache_free_blocks | 17 |
| Qcache_free_memory | 7966328 |
| Qcache_hits | 4734593 |
| Qcache_inserts | 63178 |
| Qcache_lowmem_prunes | 22354 |
| Qcache_not_cached | 340 |
| Qcache_queries_in_cache | 7933 |
| Qcache_total_blocks | 15908 |
| Questions | 236975390 |
| Rpl_status | NULL |
| Select_full_join | 0 |
| Select_full_range_join | 0 |
| Select_range | 88 |
| Select_range_check | 0 |
| Select_scan | 2077 |
| Slave_open_temp_tables | 0 |
| Slave_running | OFF |
| Slave_retried_transactions | 0 |
| Slow_launch_threads | 0 |
| Slow_queries | 543 |
| Sort_merge_passes | 1 |
| Sort_range | 10 |
| Sort_rows | 12400628 |
| Sort_scan | 1173 |
| Table_locks_immediate | 7824958 |
| Table_locks_waited | 6 |
| Threads_cached | 8 |
| Threads_connected | 8 |
| Threads_created | 39 |
| Threads_running | 2 |
| Uptime | 1042044 |
+----------------------------+------------+
157 rows in set (0.01 sec)

***********************************************************

here is the ouput from top

1.34, 1.11, 0.78
103 processes: 102 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 9.1% 0.0% 1.6% 0.2% 0.9% 49.3% 38.7%
cpu00 0.0% 0.0% 1.4% 0.0% 0.0% 40.8% 57.8%
cpu01 17.8% 0.0% 2.4% 1.0% 3.6% 55.8% 19.4%
cpu02 0.2% 0.0% 1.0% 0.0% 0.0% 41.4% 57.4%
cpu03 18.6% 0.0% 1.8% 0.0% 0.0% 59.2% 20.4%
Mem: 2060212k av, 2033544k used, 26668k free, 0k shrd, 4904k buff
1584816k actv, 193752k in_d, 25820k in_c
Swap: 522072k av, 0k used, 522072k free 1076564k cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
15937 mysql 17 0 784M 784M 1520 D 9.9 38.9 0:47 3 mysqld
606 root 15 0 0 0 0 SW 0.5 0.0 43:49 0 kjournald
11 root 15 0 0 0 0 SW 0.1 0.0 22:39 3 kswapd
1 root 15 0 116 116 64 S 0.0 0.0 0:10 1 init
2 root RT 0 0 0 0 SW 0.0 0.0 0:00 0 migration/0
3 root RT 0 0 0 0 SW 0.0 0.0 0:00 1 migration/1
4 root RT 0 0 0 0 SW 0.0 0.0 0:00 2 migration/2
5 root RT 0 0 0 0 SW 0.0 0.0 0:00 3 migration/3

Options: ReplyQuote


Subject
Views
Written By
Posted
Optimizing the MySQL server performance
1848
September 21, 2005 04:29PM
1379
September 23, 2005 01:59PM
1412
September 27, 2005 09:16AM
1427
September 29, 2005 08:23AM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.