Note that with InnoDB, the PRIMARY KEY is embedded with the data. That is, it has to rummage through the _data_ even when all it needs is the PK index. No, this does not explain what you are seeing.
Note that things are cached. Any timings should be run twice, and report the second time. This will be the 'cached' time and (usually) exclude time to load from disk. Of course once everything is in cache, there is no further need to run twice. No, this does not explain what you are seeing.
How big is the cache? innodb_buffer_pool_size. Looks like you need about 500MB just to hold all the rows for id=1. The whole table probably cannot be read without hitting the disk. This would explain "11 min 59.82 sec". But, alas, it does not explain why it is not "0.34 sec".
I feel that this is a deficiency in InnoDB, and should be reported as a bug,
http://bugs.mysql.com/ , as a missing optimization.