Re: File descriptors

[Rick James]

show variables like '%open%';
If the table cache is too small, you will get thrashing among descriptors. And each entry in the table cache uses some RAM.
The open_files_limit is derived from the OS limit.

Back on the previous thread -- DELETEing rows does not return space to the OS, but it does free up space for subsequent INSERTs. So, if you are inserting and deleting about the same number of rows each day, the disk footprint will stay pretty constant. The drawback (MyISAM) is that the new rows will be scattered around the table (usually not an issue) and may be fragmented (broken into parts, thus requiring extra disk hits). OPTIMIZE solves that last issue (but, as you noted, takes a long time).

I do not know for a fact whether JFS will have trouble with 750K files.

When doing a SELECT on a PARTITION, _all_ partitions are opened (regardless of how many are needed) -- that is, 25 (actually 50?) files have to be opened. (This is a known 'bug'; it has not yet been addressed.) Do your SELECTs only hit one partitioned table? So, probably a few SELECTs per second might not be a problem -- the thrashing of descriptors might not be "too high".

Bottom line: you may not be 'in trouble'.

Other approaches to consider: Archive. PARTITIONs of Achive. InfoBright.
All of these take much less disk space and promise fast load. And your 1-2 SELECTs/sec should not be a problem. These Engines may be significantly faster or slower for a given SELECT, depending on various things.