MySQL Forums
Forum List  »  Performance

Re: Performance trouble at 35M rows
Posted by: daofeng luo
Date: November 26, 2004 01:43AM

I store logs in MyISAM table. There are about 100 million logs(about 2G) every day, and I build the indexes on userID and login_ip. I load the logs following the steps:
1: create table log_20041124 like log_template;
2: alter table log_20041124 disable keys;
3: load data infile "***" into log_20041124;
4: alter table log_20041124 enable keys.

It takes about 30 minutes to build two B+Tree indexes.

OS: slackware Linux
MySQL: 4.1.7
Hardware: Xeon 2CPU, 2G RAM
my.cnf:
key_buffer_size 512M
max_allowed_packet = 1M
table_cache = 512
sort_buffer_size = 128M
read_buffer_size = 8M
myisam_sort_buffer_size = 128M


Ted Cui, how do you think if I store the logs in InnoDB? There will be about 4 billion logs in a month. I now use Merge table when I need to query the all logs, but the performance seems not so good. Can InnoDB handle this large volumn of data in high performance?

Options: ReplyQuote


Subject
Views
Written By
Posted
4291
November 05, 2004 11:47PM
2309
November 07, 2004 09:02PM
2393
November 10, 2004 06:55PM
2387
November 26, 2004 11:57PM
2299
November 19, 2004 04:49AM
Re: Performance trouble at 35M rows
2328
November 26, 2004 01:43AM
2351
November 27, 2004 09:21AM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.