how to optimise query for 10+ millions records
Posted by:
Azhar Rahi
Date: February 09, 2012 03:30AM
Hello How all are here.. I have recently written a query which runs perfect under the small records but gets slow when the records exceeds upto millions. The records in the table can be increased upto millions within 3,4 days. I have also applied indexes on the columns and used them in where clause, but in some cases it still scans the whole table.
So the query has a left join with another table, so that if the records does not match in the outer table, then it returns the null records. The problem is that the outer table contains a column 'DestinationName', on which we have made a filter. Well its quite possible that user want to see all destinations.So what I have have done to fulfil this, is to apply a condition with ifnull statement in where clause:
i.e.
IFNULL(destination.DestinationName,'Unknown') LIKE @l_Destination
Now when user asks for records against all destinations, it scans the whole table and hence ignore the indexes. I got this by the query anaylyzing using Expain keyword. But if I pass any keyword in @l_Destination, then the Explain shows that it has hit the indexes. So we may say that for all destinations, there can be null destinations as well, in which case the indexes are skipped.
Can anybody helps me regarding this whole scanning? So that we can avoid this.
Also we have to show only 100 records on the Page, but we are doing paging in the code, not in the database. Can it help us also to stop full scanning of the table?
Subject
Views
Written By
Posted
how to optimise query for 10+ millions records
2936
February 09, 2012 03:30AM
1724
February 09, 2012 05:31AM
1385
February 09, 2012 05:38AM
1291
February 09, 2012 05:58PM
1270
February 11, 2012 11:26PM
1525
February 12, 2012 10:12PM
1375
February 13, 2012 02:18PM
1587
February 14, 2012 02:33AM
1377
February 14, 2012 07:24AM
1397
February 15, 2012 07:30AM
1852
February 16, 2012 03:54AM
1288
February 17, 2012 01:20AM
1249
February 17, 2012 03:57PM
1526
February 17, 2012 05:12PM
1332
February 18, 2012 07:18AM
1391
February 21, 2012 03:25AM
1386
February 24, 2012 10:45AM
1330
February 20, 2012 02:21AM