Re: Data Node Won't Start: Internal program error (failed ndbrequire)

[Jeff Steele]

Here is more information about the initial crash which occurred during a backup:

2020-10-26 00:55:02 [ndbd] INFO -- Adjusting disk write speed bounds due to : Backup started
For help with below stacktrace consult:
https://dev.mysql.com/doc/refman/en/using-stack-trace.html
Also note that stack_bottom and thread_stack will always show up as zero.
stack_bottom = 0 thread_stack 0x0
/usr/sbin/ndbmtd(my_print_stacktrace(unsigned char const*, unsigned long)+0x3d) [0x8aad2d]
/usr/sbin/ndbmtd(ndb_print_stacktrace()+0x52) [0x854c62]
/usr/sbin/ndbmtd(ErrorReporter::handleError(int, char const*, char const*, NdbShutdownType)+0x2f) [0x80a99f]
/usr/sbin/ndbmtd(SimulatedBlock::progError(int, int, char const*, char const*) const+0xf9) [0x86e919]
/usr/sbin/ndbmtd(Backup::OperationRecord::scanConf(unsigned int, unsigned int, unsigned int)+0x9d) [0x7cac9d]
/usr/sbin/ndbmtd(Backup::execSCAN_FRAGCONF(Signal*)+0x19e) [0x7d9c2e]
/usr/sbin/ndbmtd() [0x87c388]
/usr/sbin/ndbmtd() [0x880abb]
/usr/sbin/ndbmtd(mt_job_thread_main+0x249) [0x885749]
/usr/sbin/ndbmtd() [0x852858]
/lib64/libpthread.so.0(+0x7ea5) [0x7fa4b63cfea5]
/lib64/libc.so.6(clone+0x6d) [0x7fa4b4df88dd]
2020-10-26 00:56:08 [ndbd] INFO -- /var/lib/pb2/sb_1-417098-1600898256.43/rpm/BUILD/mysql-cluster-gpl-8.0.22/mysql-cluster-gpl-8.0.22/storage/ndb/src/kernel/blocks/backup/Backup.cpp
2020-10-26 00:56:08 [ndbd] INFO -- BACKUP (Line: 9682) 0x00000002 Check noOfOps == done failed
2020-10-26 00:56:08 [ndbd] INFO -- Error handler shutting down system
2020-10-26 00:56:08 [ndbd] ALERT -- Node 4: Forced node shutdown completed. Caused by error 2341: 'Internal program error (failed ndbrequire)(Internal error, programming error or missing error message, please report a bug). Temporary error, restart node'.

I upgraded from version 8.0.20 to version 8.0.22 on Saturday morning and the backup ran Sunday morning without problem. But, it crashed this morning,

My main problem now, however, is that I can't start the second node.