Re: RAM consideration/ IO usage

[Jawahar muthukrishnan]

Rick - thanks for the detailed explanation.

We will be doing more INSERTS than SELECTs. Our projection is that, one of the tables will have 14 * (10^9) records at the end of two years.

1. We are considering aggregating data which are older than 3 months, which will then result in a BAD schema as you say it - * huge datasets that get purged of old data daily/monthly .

2. At the peak, we are expecting 40,000 insert requests per second. How should I account for this in the hardware configuratin/mysql configuration.

3. We are planning on using Amazon MySQL RDS instance for data storage. Our projection is at the end of 2 years, we will be storing 1.6 TB of data. Should I pick an instance with high RAM value, or look at IOPS options. Since our system requires mostly INSERTS, should IOPS be the important factor?

4. We do store lat/long values for some events, and our hope was to use "find nearest" feature of mysql.. Why is this BAD schema?

5. We do have SELECTS with count(*) but within a TIME RANGE, where I defined an index in the TIME column. THis should not result in high I/O usage right?

6. How should I test performance in MYSQL.. Shoudl I load up the database with dummy data matching our forecast, and then execute inserts/ and Selects?

7. I am being advised that MySQL will not scale, and we should seriously consider something like Redshift.. You opinion.