Mark is sharp. But I think he is wrong on two counts...
In InnoDB,
PRIMARY KEY (i), KEY (j)
is probably identical to
PRIMARY KEY (i), KEY (j, i)
Also, the "continue where you left off" is better done without the OR:
select ...
from foo
where ( j = $last_j and i < $last_i )
or j < $last_j
order by j desc, i desc
LIMIT 10;
-->
select ...
from foo
where j <= $last_j
and ( i < $last_i or j < $last_j )
order by j desc, i desc
LIMIT 10;
You won't notice any difference for small tables, but that's not what we are talking about.
Another trick is to do LIMIT 11 instead of 10. Then you can tell if there is more to be had, and can properly show or omit the "Next" link on the page.
To paginate through 100 pages of 1000 items, ten at a time, using OFFSET and LIMIT will require about 50K items read: 1000*1000/2/10.
Using the $last technique (or "left off" as I call it), it will be only 1000 items read (or maybe 1100 = 1000*11/10).