Store C++ STL Vector in MySQL Database
Posted by: A R
Date: January 22, 2012 06:39AM

Hi all

Apologies if this is in the wrong place or a badly framed question, but I'll do my best to be clear!

I have a large number of numerical values stored in CSV files. Initially I wanted to load the data into a C++ vector (or more precisely an array of vectors). But the scale of my simulations are such that the data is too big to be stored in memory (roughly 0.5 TB). Therefore I would like some solution whereby I have some representation of the data that I can quickly and easily query, and temporarily load the list of values into a C++ vector.

Explicitly my data is an array of lists of numbers of different sizes

eg

<Index> <List>
1 {1.1,2,5,3.2}
2 {1.23,3.2,6.7,7.7,8.1}
...

However there are about 6x10^6 entries and the list length is typically 10^4.

While I'm no database expert, I know the basic principles and commands of MySQL. I would just like to know if anyone has some good advice about the kind of data structure in MySQL which would suit this best. I understand that there are arrays of fixed length (but obviously I require something of variable size) and also 'blobs'?

Any guidance much appreciated.

Options: ReplyQuote


Subject
Views
Written By
Posted
Store C++ STL Vector in MySQL Database
3399
A R
January 22, 2012 06:39AM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.