Re: Maximum data size

[Ted Wennmark]

Hello,

You calculations above are correct assuming only 4GB RAM available for datamemory per datanode.

We have a limit on 48 datanodes but you can have much more memory assigned to each datanode. If you need to store TB of data in one single cluster you need to have each datanode store more data.

For example;
48 datanode with 256GB of memory each (and number of replicas set to 2) will give you the possibility to store (48x256)/2 = 6.1TB of data in the cluster.
That said you need to have some memory for OS and other buffers in cluster also.

Cheers
Ted