multiple overlapping bulk INSERTs

[Miguel Branco]

Hello,

in my application it is quite common to have very large insert requests (+80K rows) that tend to overlap a bit.

At the moment I use INSERT IGNORE with a bulk insert (using python, that means the executemany(..) method)

what I don't know from looking at the documents is what is the exact behavior of INSERT IGNORE or how different it is from doing INSERT .. ON DUPLICATE KEY UPDATE somevalue=somevalue (effectively making no change).

Also, it wasn't completely clear from the docs if INSERT IGNORE ignores the entire bulk request or only a overlapping individual row. Also, it wasn't clear if IGNORE is only triggered on duplicate key but also on other conditions ( I only want to ignore on duplicate key actually )

Any tips on how to change the app behaviour are very welcomed. Filtering the requests first (not to have overlapping inserts) is very hard because these come from multiple distributed clients, so that job is left to the database.

thanks

(and not sure if this is not the appropriate forum)