MyISAM? key_buffer_size -- maybe 20G for such, BUT -- there is a limit of 4GB per key buffer, so ponder whether multiple key buffers makes sense.
InnoDB? innodb_buffer_pool_size -- perhaps 90G
More than 4 cores? Turn off cores to get it down to 4. Or use 5.4 or the "Google fixes" or Percona's version or Virident.
Query Cache?
Plan A: OFF
Plan B:
* No more than, say, 200M. Why? The code is not well optimized to handle huge amount of space.
* DEMAND
* Carefully pick which SELECTs say SQL_CACHE
What's your write:read ratio? If it is 90:10 or higher, consider Virident; it can give you 250GB of DRAM-like Flash memory.
Is the machine dedicated to MySQL? If not, consider separating functions between servers.
Read scaling needed? Consider Master-Slave setup. Any number of slaves can be used.
Tuning is rarely as useful as optimizing queries. Do you have control over the Indexes and the queries?
Sharding: By separating the data along some key, you can have multiple machines, each with 'smaller' RAM but better cacheability.
http://forums.mysql.com/read.php?10,275221 (Help me Optimize my my.ini [Confuzed])
http://forums.mysql.com/read.php?24,260895 (Simple Performance Question)
http://forums.mysql.com/read.php?10,269653 (Help with friendship table - Social network development)
http://forums.mysql.com/read.php?22,266939 (innodb or myisam for logging)
http://forums.mysql.com/read.php?10,263717 (Strange query speed issues)
http://forums.mysql.com/read.php?20,263480 (Regarding use of swap memory for MySQL.)
http://forums.mysql.com/read.php?10,262139 (Page File Bytes)
http://forums.mysql.com/read.php?24,262644 (MYSQL eating up CPU!)
http://forums.mysql.com/read.php?10,260695 (What Happens To InnoDB Table When A Query Is Crashed)
http://forums.mysql.com/read.php?10,258390 (High CPU use (spikes) and Page Faults in VM)
http://forums.mysql.com/read.php?21,25724 (Storage capacity)