Re: NDB engine possibly not a realtime database engine?

[Mikael Ronström]

I know of no way of proving or disproving that it is the OS.
Maybe there is some tool that tracks scheduling stats for a thread?
Here is one idea of how you can track this:
1) Create a script that looks for the printouts in the data node log.
As far as I remember these printouts starts already at 100ms sleep
time, so your script have almost 6 seconds to react to analyse the
situation.

2) When the NDB kernel thread is stuck happens, then use DTrace scripts
to discover what is going on.
I think you should be able to get a stack trace of the thread that is
stuck using DTrace and you can probably also get some similar information
about what the OS is doing. If your thread is locked to a CPU, then you
can probably also get a stack trace of what the CPU is currently doing.

3) The data node log at startup prints the thread id of each thread, so
using this information you can map the number of thread to the
thread id. Also the printout at startup clearly specifies what CPU the
thread is locked onto, if any.

Hopefully this can give you a path to the solution.

Have not heard of any similar problems, so hard to say what might
be the actual problem. The only possible problem that could be an
internal NDB problem would be some sort of loop. But in this case
it is more or less always the case that the loop is eternal (which
is the reason we have those watchdog threads to catch any such
problems).