Re: Debugging Memory Engine Problem

[Rick James]

Several thoughts, none of them certain...

You have 240GB or RAM, correct?

Do this before the CREATE TABLE, just to verify that the settings took hold:
SHOW VARIABLES LIKE '%table_size';

Consider doing the CREATE this way:
CREATE TABLE points_mem (val int, x int, y int,
INDEX(x,y) USING BTREE
) ENGINE = MEMORY MAX_ROWS=5000000000;

What are the values here?
SHOW VARIABLES LIKE '%buffer%';

By having tmp_table_size = 150GB, it may be allocating that much to do the CREATE INDEX. Decreasing that may actually help.