Hope someone can help, this is my first post.
Im getting an out of memory error trying to store large blobs in a mysql database. I think I have tracked the issue down to connector j.
I have code like this to set the blob field with a java.io.InputStream:
InputStream fstream = new FileInputStream("C:\\c.zip");
int a = fstream.available();
m_PreStmt.setBinaryStream(5,fstream,a);
When the file size gets over 100meg, or 200 meg; I get an out of memory error.
My system has over a gig of memory, and I have the JVM set to 500meg.
My understanding of setting a blob to an inputstream is that this should avoid the
whole (chunk of data) being loaded into memory at once... that the stream would
be read a little at a time.
I have tried several versions of connector j 3.0.11, 3.0.1.5.ga, and 3.1.4-beta, all give the same out of memory error.
In the source for connector j 3.1.4-beta the error occurs in the method ensureCapacity(int additionalData)
The line of code is: byte[] newBytes = new byte[newLength];
It seems that the entire chunk of memory is being allocated?
Can anyone offer any help?
also this seems to be the same type problem as reported in bug # 1605
http://bugs.mysql.com/bug.php?id=1605
Thank You!
final void ensureCapacity(int additionalData) throws SQLException {
if ((this.position + additionalData) > getBufLength()) {
if ((this.position + additionalData) < this.byteBuffer.length) {
// byteBuffer.length is != getBufLength() all of the time
// due to re-using of packets (we don't shrink them)
//
// If we can, don't re-alloc, just set buffer length
// to size of current buffer
setBufLength(this.byteBuffer.length);
} else {
//
// Otherwise, re-size, and pad so we can avoid
// allocing again in the near future
//
int newLength = (int) (this.byteBuffer.length * 1.25);
if (newLength < (this.byteBuffer.length + additionalData)) {
newLength = this.byteBuffer.length
+ (int) (additionalData * 1.25);
}
if (newLength < this.byteBuffer.length) {
newLength = this.byteBuffer.length + additionalData;
}
---> byte[] newBytes = new byte[newLength];
System.arraycopy(this.byteBuffer, 0, newBytes, 0,
this.byteBuffer.length);
this.byteBuffer = newBytes;
setBufLength(this.byteBuffer.length);
}
}
}