Hello MySQL Gurus,
I am writing a C program which is a data translator between a Message Queue server and a MySQL server. The C code is hitting what I believe to be Bug 1434758; an issue with too many open connections/file descriptors.
First, some environmental information:
• The C code runs on an Ubuntu box, running 16.04.4
• My C compiler is GCC 5.4.0
• I’m using MySQL package Ver 14.14 Distrib 5.7.26 with my C code
• And the remote MySQL server is a docker container (mysql:8.0.16)
Okay, on to my code. The C program does the following:
• Upon startup, the program listens on TCP 12345 and also opens a connection to the remote MySQL server.
• When the remote message server sends a data message, the C program parses that message, and converts it into a MySQL INSERT command.
• The C program then forwards the MySQL INSERT command to the MySQL server.
• The program is meant to run forever
When I stress-test the C program, I successfully get about 1000 requests to the MySQL server. Great! But then, the C program suddenly starts reporting ERRNO 24 for every new incoming message, and everything grinds to a halt.
(For C programmers, the ERRNO 24 is thrown by the accept() system call.)
After some digging, I discovered Bug 1434758. (Some discussion here:
https://bugs.launchpad.net/ubuntu/+source/mysql-5.6/+bug/1434758) It seems that in 2015, this was a known bug with the MySQL libraries. Essentially, every time a program sent a MySQL server a query, the program’s system considered the connection live, long after the query was completed. The workaround was to simply allow higher number of connections, in the hope that you would never run out.
But I fear that will not work for me. My C program will need to run for a long, long, loooooooooooong time, so any finite ceiling will still be an eventual showstopper for me.
Does anyone know if this was fixed, and if so, what the fix might have been? I haven’t found anything, but I can’t believe I’m the first person to encounter this problem. Much appreciated.