MySQL Forums
Forum List  »  Newbie

Re: Counters
Posted by: Rick James
Date: March 15, 2009 11:19AM

For cases like this, I like to "meet in the middle". Let's build a simple summary table that has counts, totals, maxes, etc for each day. The report writer will pick whatever date range it likes. The nightly job, and the report job, each do some of the arithmetic. Store only daily stuff, not monthly/yearly, you can easily and quickly calculate monthly/yearly stuff on demand.

Just after midnight add a row(s) to a permanent table 'counters':
   INSERT INTO counters
      SELECT date,
             xyz count(*) as ct,
             sum(...) as sum_foo,
             max(abc) as max_abc_for_day
             etc
         FROM raw_data
         WHERE date is yesterday
         GROUP BY xyz;
This would be indexed by (date, xyz)

For reporting:
    SELECT xyz,
           sum(ct) as 'number of xyzs',
           sum(sum_foo) / sum(ct) as 'average foo',
           max(max_abc_for_day) as 'Max abc for Year',
           etc
         FROM counters
         WHERE date BETWEEN DATE_SUB(now(), 1 YEAR) AND now()
         GROUP BY xyz;
And you can change the BETWEEN to be any range you like - last 7 days, previous week, year-to-date, etc. They all work from the same "counters" table.

This will be plenty fast for almost any application like yours.

No temp tables needed (in this example). UPDATEing existing counters is not a good solution (my opinion).

Using GROUP BY, your report could give monthly values for the past year, all in one SELECT.

My sample code breaks down things by day and xyz. If you need different breakdowns, create more summary tables. But remember that "week" can be derived from day, so you don't need an extra table for that. And a report that ignores "xyz" can also use the same table. (Since I don't quite grok what "section" and "Story_Number" are, I am not using your terms.)

More discussion:
http://forums.mysql.com/read.php?125,245133,245357

Options: ReplyQuote


Subject
Written By
Posted
March 14, 2009 02:47AM
Re: Counters
March 15, 2009 11:19AM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.