Skip navigation links

MySQL Forums :: Connector/JDBC and Java :: useCompression=true and potential Memory leak due to connection tracking via CompressionInputStreamMemoryLeak


Advanced Search

useCompression=true and potential Memory leak due to connection tracking via CompressionInputStreamMemoryLeak
Posted by: Dominic Tootell ()
Date: May 12, 2013 12:27PM

Hi there,

We came across an issue on one of our systems towards the end of last week, where the application was growing in memory use. Upon analysing the hprof of a couple of replica systems, it was noticed that there seemed to be a consistency across the hprof's with that of a growing number of JDBC4Connection objects in memory. For a proportion of those JDBC4Connections, the "forceClosedReason" was filled out; and as a result the JDBC4Connection shouldn't really be strong referenced anymore.

The only class that seemed to have strong reference to the JDBC4Connection was that of the static map "connectionPhantomRefs" in NonRegisteringDriver and the associated ConnectionPhantomReference.

The date the issue of increasing memory usage (akin to a Memory Leak) also co-incided with our upgrade to 5.1.23 of Connector/J.

----

I've spend some time trying to understand where the issue is occurring, and I beleive I have managed to replicate the issue in a sample piece of code; removing the application code from the equation.

From what seems to be occurring is that when "useCompression=true" is enabled, the NonRegisteringDriver.trackConnection(this) is holding onto the Connection; even though the application no longer has a strong reference to the Connection. With "useCompression=false", the Connections are not held onto and the Leak does not occur.

From what it looks like is that the CompressionInputStream is keeping the JDBC4Connection object alive, for which the ConnectionPhantomReference is monitoring for pre-gc clean up. The CompressionInputSteam itself has a reference to the JDBC4Connection, and the ConnectionPhantomReference has a strong reference to the CompressionInputStream via the NetworkResource object:



ConnectionPhantomReference ---> NetworkResources

NetworkResources ---> io (MysqlIO) ----> CompressedInputStream

CompressedInputStream ---> JDBC4Connection




I think this might be solvable via the use of a WeakReference in CompressedInputStream, but I don't know if this use of a WeakReference may cause regression elsewhere.

I've attempted to document the investigation into the issue here:

https://github.com/tootedom/CompressionInputStreamMemoryLeak

With the sample code to replicate it. I hope the write up/investigation makes sense, and that I've not over looked something, but apologies if I have indeed gone down the wrong track on this.

Is anyone else witnessing this slow growth in the number of connections in the heap? Or is this issue already known about?


Thanks in advance for any assistance. Please let me know if I've mis-represented anything, or if anything requires clarifying.


cheers
/dom

Options: ReplyQuote


Subject Written By Posted
useCompression=true and potential Memory leak due to connection tracking via CompressionInputStreamMemoryLeak Dominic Tootell 05/12/2013 12:27PM
Re: useCompression=true and potential Memory leak due to connection tracking via CompressionInputStreamMemoryLeak Alexander Soklakov 05/13/2013 04:10AM
Re: useCompression=true and potential Memory leak due to connection tracking via CompressionInputStreamMemoryLeak Dominic Tootell 05/13/2013 06:07AM
Re: useCompression=true and potential Memory leak due to connection tracking via CompressionInputStreamMemoryLeak Dominic Tootell 05/13/2013 07:56AM
Re: useCompression=true and potential Memory leak due to connection tracking via CompressionInputStreamMemoryLeak Alexander Soklakov 05/13/2013 10:32AM
Re: useCompression=true and potential Memory leak due to connection tracking via CompressionInputStreamMemoryLeak Dominic Tootell 05/13/2013 11:31AM
Re: useCompression=true and potential Memory leak due to connection tracking via CompressionInputStreamMemoryLeak Mark Matthews 05/14/2013 07:54AM
Re: useCompression=true and potential Memory leak due to connection tracking via CompressionInputStreamMemoryLeak Mark Matthews 05/14/2013 08:16AM
Re: useCompression=true and potential Memory leak due to connection tracking via CompressionInputStreamMemoryLeak Alexander Soklakov 05/14/2013 08:06AM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.