MySQL Forums
Forum List  »  Performance

Re: Performance trouble at 35M rows
Posted by: daofeng luo
Date: November 26, 2004 01:43AM

I store logs in MyISAM table. There are about 100 million logs(about 2G) every day, and I build the indexes on userID and login_ip. I load the logs following the steps:
1: create table log_20041124 like log_template;
2: alter table log_20041124 disable keys;
3: load data infile "***" into log_20041124;
4: alter table log_20041124 enable keys.

It takes about 30 minutes to build two B+Tree indexes.

OS: slackware Linux
MySQL: 4.1.7
Hardware: Xeon 2CPU, 2G RAM
key_buffer_size 512M
max_allowed_packet = 1M
table_cache = 512
sort_buffer_size = 128M
read_buffer_size = 8M
myisam_sort_buffer_size = 128M

Ted Cui, how do you think if I store the logs in InnoDB? There will be about 4 billion logs in a month. I now use Merge table when I need to query the all logs, but the performance seems not so good. Can InnoDB handle this large volumn of data in high performance?

Options: ReplyQuote

Written By
November 05, 2004 11:47PM
November 07, 2004 09:02PM
November 10, 2004 06:55PM
November 26, 2004 11:57PM
November 19, 2004 04:49AM
Re: Performance trouble at 35M rows
November 26, 2004 01:43AM
November 27, 2004 09:21AM

Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.