innodb memory usage vs swap

[kevin foote]

Hi I have a question about keeping mysql using innodb engine out of swap.

I see this from top output:

PID USER PR NI VIRT SWAP RES SHR S %CPU %MEM TIME+ COMMAND
3194 mysql 0 -15 5272m 2.9g 2.3g 6568 S 31.2 30.1 1525:28 mysqld


This is saying half my innodb buffer is in swap?? This can't be good.

Here are the important parts of my.cnf on this host (RHEL5 x86_64, 8G mem)

------------- >%
# Log
log-error = /var/log/mysqld/mysqld.log
log_warnings = 2
#slow_query_log = 1
slow_query_log_file = /var/log/mysql/mydb2-slow.log

# Buffers and Connections
key_buffer = 2M
max_allowed_packet = 32M
sort_buffer_size = 1M
read_buffer_size = 1M
query_cache_size = 128M
connect_timeout = 60
max_connections = 350
max_connect_errors = 1844674407370954751
wait_timeout = 300
interactive_timeout = 300

## INNODB SETTINGS
innodb_data_home_dir = /var/lib/mysql/ibdata
innodb_data_file_path = ib_data1:100M:autoextend
innodb_log_group_home_dir = /var/lib/mysql/ibdata_logs
innodb_log_file_size = 256M
innodb_log_files_in_group = 2
innodb_log_buffer_size = 32M
innodb_buffer_pool_size = 4G
innodb_additional_mem_pool_size = 20M
innodb_support_xa = 1
innodb_flush_log_at_trx_commit = 1
innodb_flush_method = O_DIRECT
------------------------------------------ >%

I've also set a few tunables in /etc/sysctl.conf

kernel.shmmax = 68719476736
kernel.shmall = 429496
vm.nr_hugepages = 2048

I've seen two huge swap outs today which really spins up the thread count coming from my app-servers.

Any pointers on keeping this out of swap would be much appreciated..

thanks -kevin