MySQL 5.5. Scalability

[Anh Nguyen]

Hi,

I am running a mixed load test using "Server version: 5.5.1-m2-community MySQL Community Server (GPL)", various data points below suggest I am having scalability issues or other bottlenecks, and perhaps related to the front-end.
Please review and confirm my observations.
Are there any tuning knobs related to the front-end I should consider?
I do have some questions interspersed below too. Hope you could help clarify.
Thanks

Anh-

>>> Only 4% of my transactions are active:
[perf14] /lipaperf/pt3/log$ # grep "not started" dbstatAt600.out | wc -l
300
[perf14] /lipaperf/pt3/log$ # grep ACTIVE dbstatAt600.out | wc -l
13
[perf14] /lipaperf/pt3/log$ #

>>> Most IO threads are in "waiting for requests" state:

--------
FILE I/O
--------
I/O thread 0 state: waiting for i/o request (insert buffer thread)
I/O thread 1 state: waiting for i/o request (log thread)
I/O thread 2 state: waiting for i/o request (read thread)
I/O thread 3 state: waiting for i/o request (read thread)
I/O thread 4 state: waiting for i/o request (read thread)
I/O thread 5 state: waiting for i/o request (read thread)
I/O thread 6 state: waiting for i/o request (write thread)
I/O thread 7 state: waiting for i/o request (write thread)
I/O thread 8 state: waiting for i/o request (write thread)
I/O thread 9 state: waiting for i/o request (write thread)


Threads_cached 0 <<< Does this mean all threads in the cached have been used???
Threads_connected 401
Threads_created 401
Threads_running 2 <<< Why so low?????




>>> Profiling data

samples % image name app name symbol name
333466 8.7563 stap_989e9d1424751d669bcdfeb2edbc11d9_21121 stap_989e9d1424751d669bcdfeb2edbc11d9_21121 /stap_989e9d1424751d669bcdfeb2edbc11d9_21121
220841 5.7990 mysqld mysqld MYSQLparse(void*)
142079 3.7308 s2io s2io /s2io
132029 3.4669 vmlinux vmlinux pre_handler_kretprobe
111359 2.9241 mysqld mysqld MYSQLlex(voi