Communications link failure due to underlying exception...various network exceptions

[mreichman]

Hi,

I've seen similar messages to this, but not with anything this recent. Using MySQL 5.0.27, Connector/J 5.0.4, and Tomcat 5.5.20. We're seeing random SocketException connections dropping, and the amount has increased significantly (once every week or so) since we moved from 4.1.11 and Connector/J 3.1.8. Here's one exception with one type of java.net.SocketException:
com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:

** BEGIN NESTED EXCEPTION **

java.net.SocketException
MESSAGE: Connection reset by peer: socket write error
STACKTRACE:
java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:2637)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1554)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1665)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3176)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1153)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1266)
at com.virage.util.database.SQLProcessor.handleQuery(SQLProcessor.java:398)
at com.virage.util.database.SQLProcessor.doExecuteQuery(SQLProcessor.java:213)
at com.virage.util.database.SQLProcessor.executeQuery(SQLProcessor.java:171)
at com.virage.store.db.ItemLoader.loadItems(IndexSchemaLoader.java:293)
at com.virage.store.db.IndexSchemaLoader.load(IndexSchemaLoader.java:103)
at com.virage.store.db.IndexManagerImpl.loadIndexListFromDB(IndexManagerImpl.java:207)
at com.virage.store.db.IndexManagerImpl.getIndexMap(IndexManagerImpl.java:185)
at com.virage.store.db.DbAssetLoader$AssetIndexLabelSetRP.process(DbAssetLoader.java:290)
at com.virage.util.database.SQLProcessor.handleQuery(SQLProcessor.java:407)
at com.virage.util.database.SQLProcessor.doExecuteQuery(SQLProcessor.java:213)
at com.virage.util.database.SQLProcessor.executeQuery(SQLProcessor.java:171)
at com.virage.store.db.DbAssetLoader.loadIndexLabelSetNames(DbAssetLoader.java:327)
at com.virage.store.db.DbAssetLoader.load(DbAssetLoader.java:109)
at com.virage.store.db.AssetManagerImpl.getAssetList(AssetManagerImpl.java:89)
at com.virage.store.db.AssetManagerImpl.getAsset(AssetManagerImpl.java:60)
at com.virage.vs.webservices.VsTrickleImpl.processAssetTransaction(VsTrickleImpl.java:176)
at sun.reflect.GeneratedMethodAccessor126.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.axis.providers.java.MsgProvider.processMessage(MsgProvider.java:141)
at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:453)
at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:833)
at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:639)
at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1285)
at java.lang.Thread.run(Thread.java:595)

** END NESTED EXCEPTION **

Last packet sent to the server was 0 ms ago.

Here's another, with a different java.net.SocketException:
** BEGIN NESTED EXCEPTION **
com.mysql.jdbc.CommunicationsException
MESSAGE: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.net.SocketException
MESSAGE: Software caused connection abort: socket write error
STACKTRACE:
java.net.SocketException: Software caused connection abort: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:2637)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1554)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1665)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3176)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1153)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1266)
at com.virage.util.database.SQLProcessor.handleQuery(SQLProcessor.java:398)
at com.virage.util.database.SQLProcessor.doExecuteQuery(SQLProcessor.java:213)
at com.virage.util.database.SQLProcessor.executeQuery(SQLProcessor.java:171)
at com.virage.store.db.IndexSchemaLoader.load(IndexSchemaLoader.java:100)
at com.virage.store.db.IndexManagerImpl.loadIndexListFromDB(IndexManagerImpl.java:207)
at com.virage.store.db.IndexManagerImpl.getIndexMap(IndexManagerImpl.java:185)
at com.virage.store.db.DbAssetLoader$AssetIndexLabelSetRP.process(DbAssetLoader.java:290)
at com.virage.util.database.SQLProcessor.handleQuery(SQLProcessor.java:407)
at com.virage.util.database.SQLProcessor.doExecuteQuery(SQLProcessor.java:213)
at com.virage.util.database.SQLProcessor.executeQuery(SQLProcessor.java:171)
at com.virage.store.db.DbAssetLoader.loadIndexLabelSetNames(DbAssetLoader.java:327)
at com.virage.store.db.DbAssetLoader.load(DbAssetLoader.java:109)
at com.virage.store.db.AssetManagerImpl.getAssetList(AssetManagerImpl.java:89)
at com.virage.store.db.AssetManagerImpl.getAsset(AssetManagerImpl.java:60)
at com.virage.atg.util.AssetUtil.<init>(AssetUtil.java:68)
at com.virage.atg.cars.plugins.TranslationPlugIn.populateAssetData(TranslationPlugIn.java:545)
at com.virage.atg.cars.plugins.TranslationPlugIn.run(TranslationPlugIn.java:54)
at com.virage.atg.cars.PlugInManager.run(PlugInManager.java:196)
at com.virage.atg.thread.ATGThread.doWork(ATGThread.java:117)
at com.virage.atg.thread.ATGThread.run(ATGThread.java:142)

** END NESTED EXCEPTION **

These connections don't ever remain idle as far as I know, but even if they were, did something change in that regard between 4.1.11/3.1.7 and 5.0.27/5.0.4? What else would be causing these socket exceptions?

my wait_timeout is set to 28800. If there are other relevant variables, please let me know and I'll post them.

Thanks!