MySQL Forums
Forum List  »  Performance

Re: Filtering data from BLOB on server?
Posted by: Rick James
Date: November 19, 2013 06:26PM

> would mean gather 256x256 points at a given slice positon, but i may also need the full stack of 1024 values at a given point

So, you have a 3-dimensional object, and you want to extract (slice) planes or lines out of it. (If it is only 2D, the concepts still apply. I don't understand what you mean by "256x256x1024", so I may not be tracking you quite right.)

You can optimize for one or the other, not both. The following optimization boils down to "counting the disk hits".

If a 'slice' is stored as consecutive bytes on disk, then it is less I/O. But slices in different dimensions won't be optimal this way.

If you want to optimize for fetching a 256x256x2 byte slice, then have 1024 rows, each with a MEDIUMBLOB with 128KB. Fetch one row to fetch that one slice. That will be on the order of 1024 times as fast as fetching the 128MB and extracting the slice. (Because of overhead, it won't be a full 1024x faster.)

But that does nothing good for finding the 1024x2 bytes for one pixel.

Well... You could build 3(?) tables, one for each slicing that you want. This would involve writing the data 3 times, but the SELECTs would be _much_ faster.

However,... If your processing involves updating pixel(s), that gets really messy, since you need to update all 3 tables.

Another thought... Try compressing the 128MB. If it is heavily compressible (.bmp files are very compressible), then the cost of compression/uncompression may be less than the cost of the I/O.

Both suggestions (pre-slicing and compression) may work well together.

Options: ReplyQuote


Subject
Views
Written By
Posted
2283
E L
November 17, 2013 05:05PM
Re: Filtering data from BLOB on server?
1030
November 19, 2013 06:26PM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.