mysql-connector-python starting from 8.0.12 goes to infinite loop when iterates till the end of table
I have CentOS 7.8, Python 3.6.8 and Django 2.2.12 installed via pip.
When I install mysql-connector-python 8.0.6 or 8.0.5 (also via pip), everything works fine.
But for versions 8.0.12, 14, and 19, when I use Django queryset to iterate through all the rows of a simple table (with 20 fields and 200 rows), it goes to an infinite loop and takes all my RAM (about 2GB) and then crashes.
Here are examples how the stack trace can look after several seconds of executing (in 8.0.6 it takes much less than a second to finish this loop):
>>> for r in Club.objects.all():
... pass
...
^CTraceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 272, in __iter__
self._fetch_all()
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 1179, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 64, in __iter__
obj = model_cls.from_db(db, init_list, row[model_fields_start:model_fields_end])
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/base.py", line 507, in from_db
new = cls(*values)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/base.py", line 497, in __init__
post_init.send(sender=cls, instance=self)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/dispatch/dispatcher.py", line 178, in send
for receiver in self._live_receivers(sender)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/dispatch/dispatcher.py", line 260, in _live_receivers
receiver = receiver()
File "/usr/lib64/python3.6/weakref.py", line 67, in __call__
obj = super().__call__()
KeyboardInterrupt
^CTraceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 274, in __iter__
self._fetch_all()
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 55, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1173, in execute_sql
return list(result)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in cursor_iter
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in <lambda>
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/utils.py", line 96, in inner
return func(*args, **kwargs)
File "/usr/local/venv/main/lib64/python3.6/site-packages/mysql/connector/cursor_cext.py", line 505, in fetchmany
if self._nextrow and self._nextrow[0]:
KeyboardInterrupt
^CTraceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 274, in __iter__
self._fetch_all()
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/query.py", line 55, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1173, in execute_sql
return list(result)
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in cursor_iter
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1552, in <lambda>
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
File "/usr/local/venv/main/lib64/python3.6/site-packages/django/db/backends/utils.py", line 22, in __getattr__
def __getattr__(self, attr):
KeyboardInterrupt
This problem appears only if I iterate to the end of the table. My table has ~300 rows, and there are no mistakes when iterating through Club.objects.all()[:100], but there is an infinite loop when iterating through any of:
Club.objects.all()
Club.objects.all()[:1000]
Club.objects.all()[5:]
I also tried 8.0.11 but I got a different error: 'datetime.date' object has no attribute 'split'. Versions 8.0.7-8.0.10 aren't available in pip.
Any ideas why this can happen?