MySQL Forums
Forum List  »  NDB clusters

Too many NDB objects with 3 billion+ rows
Posted by: Matti Oikarinen
Date: September 27, 2012 11:39AM

Hi

I'm getting error too many NDB objects when trying to write / update data in cluster which is supposed to have still free space. Is there any undocumented limits on total number of objects in cluster or did I miss something obvious? I have written over 3billion objects to 2 tables, but modifying/adding start to give this error. It goes away for a while after cluster restart, but then comes back.

Here are my specs:
* 6 NDBD nodes, using Amazon High IO instances with SSDs on raid 0
* Data files have free extents available on all nodes
* Amount of partitions was increased to fit all data
* ~1 billion rows on table with disk storage, hash index on primary key only.
* ~2 billion rows on table with memory storage, hash index on primary key only.
* Error starts to occur after updating rows in table with disk storage and goes away with cluster restart (until re-appears again after few hundred thousand writes). After error starts to occur it impacts all queries.

Errors from ClusterJ are:
SEVERE: Error in NdbJTie: returnCode -1, code 4,105, mysqlCode -1, status 2, cla
ssification 1, message Too many Ndb objects .


Memory usage:
Connected to Management Server at: localhost:1186
Node 3: Data usage is 92%(1388772 32K pages of total 1501824)
Node 3: Index usage is 87%(880319 8K pages of total 1007104)
Node 4: Data usage is 90%(1356214 32K pages of total 1501824)
Node 4: Index usage is 85%(857617 8K pages of total 1007104)
Node 5: Data usage is 90%(1356221 32K pages of total 1501824)
Node 5: Index usage is 85%(858343 8K pages of total 1007104)
Node 6: Data usage is 91%(1369744 32K pages of total 1501824)
Node 6: Index usage is 86%(869930 8K pages of total 1007104)
Node 7: Data usage is 90%(1357960 32K pages of total 1501824)
Node 7: Index usage is 85%(863422 8K pages of total 1007104)
Node 8: Data usage is 89%(1350171 32K pages of total 1501824)
Node 8: Index usage is 85%(856716 8K pages of total 1007104)


Configuration of NDBD:

[NDBD DEFAULT]
# Note: 1 replica on purpose in order to save costs for initial tests
NoOfReplicas=1
Datadir=/storage/ndbd/
FileSystemPathDD=/storage/ndbd/
BackupDataDir=/storage/ndbd//backup/
DataMemory=46932M
IndexMemory=7867M
LockPagesInMainMemory=1
MaxNoOfConcurrentOperations=1M
MaxNoOfConcurrentTransactions=16384
StringMemory=25
MaxNoOfTables=4096
MaxNoOfOrderedIndexes=2048
MaxNoOfUniqueHashIndexes=512
MaxNoOfAttributes=24576
MaxNoOfTriggers=14336
FragmentLogFileSize=256M
InitFragmentLogFiles=SPARSE
NoOfFragmentLogFiles=184
RedoBuffer=64M
TransactionBufferMemory=8M
TimeBetweenGlobalCheckpoints=20
TimeBetweenEpochs=100

TimeBetweenEpochsTimeout=32000

DiskCheckpointSpeedInRestart=100M
DiskCheckpointSpeed=10M
TimeBetweenLocalCheckpoints=20

HeartbeatIntervalDbDb=1500
HeartbeatIntervalDbApi=1500

TimeBetweenWatchdogCheckInitial=60000
RedoOverCommitCounter=3
RedoOverCommitLimit=20
SharedGlobalMemory=384M
DiskPageBufferMemory=640M

MaxNoOfExecutionThreads=8

LongMessageBuffer=32M
BatchSizePerLocalScan=512


- Matti



Edited 2 time(s). Last edit at 09/27/2012 12:32PM by Matti Oikarinen.

Options: ReplyQuote


Subject
Views
Written By
Posted
Too many NDB objects with 3 billion+ rows
2009
September 27, 2012 11:39AM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.