Regarding "Communications link failure" in MYSQL -5.5.21 and Jboss 5.1.2
Posted by: Imran Jboss
Date: July 03, 2013 09:38AM

Hello Folks,

Could you please help me on the below issue?

Event: While doing AP → DB process, there is a process which run SQL for long time, when passing 3600 seconds(60 minutes) then the session was disconnected and an error occurred in Jboss side then after that it became the status which the process was not running normally unless rebooting the instance of Jboss. No issues if it is processing within 3600 seconds.

Error in Jboss Log:

***************************************************************

2013-01-28 16:13:29,301 WARN [org.apache.commons.validator.ValidatorResources] (ajp-XXXXXX%2F137.153.101.96-8009-7) Form 'XXX030021Form' not found for locale 'ja_JP'
2013-01-28 16:15:11,783 WARN [org.jboss.resource.connectionmanager.TxConnectionManager] (ajp-XXXXXX%2F137.153.101.96-8009-3) Connection error occured: org.jboss.resource.connecti
onmanager.TxConnectionManager$TxConnectionEventListener@710feed9[state=NORMAL mc=org.jboss.resource.adapter.jdbc.local.LocalManagedConnection@141c8ac5 handles=1 lastUse=1359353701119 permit=true track
ByTx=false mcp=org.jboss.resource.connectionmanager.JBossManagedConnectionPool$OnePool@5dda95d3 context=org.jboss.resource.connectionmanager.InternalManagedConnectionPool@39a27fc8 xaResource=org.jboss
.resource.connectionmanager.TxConnectionManager$LocalXAResource@5b734107 txSync=null]
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 3,610,253 milliseconds ago. The last packet sent successfully to the server was 3,610,252 milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3102)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2991)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3532)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2002)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2163)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2624)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2127)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2293)
at org.jboss.resource.adapter.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:342)
at XXXX.XXX.common.db.XXXNamedPreparedStatement.executeQuery(XXXNamedPreparedStatement.java:252)
at XXXX.XXX.common.db.XXXDbOperator.executeQuery(XXXDbOperator.java:317)
at XXXX.XXX.XXX01info.dao.impl.XXX010010DaoImpl.search(XXX010010DaoImpl.java:826)
at XXXX.XXX.XXX01info.logic.XXX010010Logic.search(XXX010010Logic.java:224)
at XXXX.XXX.XXX01info.action.XXX010010Action.action(XXX010010Action.java:70)
at XXXX.XXX.common.action.XXXAction.execute(XXXAction.java:365)
at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
at XXXX.XXX.common.processor.XXXStrutsRequestProcessor.process(XXXStrutsRequestProcessor.java:120)
at XXXX.XXX.common.servlet.XXXActionServlet.process(XXXActionServlet.java:146)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:462)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at XXXX.XXX.common.filter.XXXEncodingFilter.doFilter(XXXEncodingFilter.java:60)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:183)
at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:95)
at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.jboss.web.tomcat.service.request.ActiveRequestResponseCacheValve.internalProcess(ActiveRequestResponseCacheValve.java:74)
at org.jboss.web.tomcat.service.request.ActiveRequestResponseCacheValve.invoke(ActiveRequestResponseCacheValve.java:47)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:385)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:451)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:185)
at com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:114)
at com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:161)
at com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:189)
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2549)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3002)
... 45 more
2013-01-28 16:15:11,785 WARN [XXXX.XXX.common.db.XXXNamedPreparedStatement] (ajp-XXXXXX%2F137.153.101.96-8009-3) [ip:43.29.15.247] (SQL??)<<SQL:XXX010010DaoImpl.s
earch>> [ErrorCode:0] [SQLState:08S01] [Message:Communications link failure

The last packet successfully received from the server was 3,610,253 milliseconds ago. The last packet sent successfully to the server was 3,610,252 milliseconds ago.]
2013-01-28 16:15:11,787 WARN [XXXX.XXX.common.servlet.XXXActionServlet] (ajp-XXXXXX%2F137.153.101.96-8009-3) [ip:43.29.15.247] [elapsed time:3,610,312ms] [pid:XXX01
0010] [aid:search] [uri:/arcstar/XXX010010.*] [ip:43.29.15.247]
2013-01-28 16:15:13,630 ERROR [XXXX.XXX.common.servlet.XXXLoginServlet] (ajp-XXXXXX%2F137.153.101.96-8009-3) [ip:43.29.72.226] ?????????:Connection is not a
ssociated with a managed connection.org.jboss.resource.adapter.jdbc.jdk6.WrappedConnectionJDK6@433707e2
java.sql.SQLException: Connection is not associated with a managed connection.org.jboss.resource.adapter.jdbc.jdk6.WrappedConnectionJDK6@433707e2
at org.jboss.resource.adapter.jdbc.WrappedConnection.lock(WrappedConnection.java:97)
at org.jboss.resource.adapter.jdbc.WrappedConnection.prepareStatement(WrappedConnection.java:254)
at XXXX.XXX.common.db.XXXNamedPreparedStatement.<init>(XXXNamedPreparedStatement.java:197)
at XXXX.XXX.common.db.XXXNamedPreparedStatement.<init>(XXXNamedPreparedStatement.java:128)
at XXXX.XXX.common.db.XXXDbOperator.executeQuery(XXXDbOperator.java:311)
at XXXX.XXX.common.db.XXXDbOperator.getBeanList(XXXDbOperator.java:757)
at XXXX.XXX.common.util.XXXPrivilegeUtil.getSpecialAuthorityGrps(XXXPrivilegeUtil.java:96)
at XXXX.XXX.common.util.XXXResourceUtility.getSpecialAuthority(XXXResourceUtility.java:318)
at XXXX.XXX.common.util.XXXRolesUtil.setSpecialAuthority(XXXRolesUtil.java:2043)
at XXXX.XXX.common.util.XXXAwsUtil.setUserInfo(XXXAwsUtil.java:234)
at XXXX.XXX.common.servlet.XXXLoginServlet.doXXXPost(XXXLoginServlet.java:183)
at XXXX.XXX.common.servlet.XXXLoginServlet.doPost(XXXLoginServlet.java:82)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:183)
at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:95)
at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.jboss.web.tomcat.service.request.ActiveRequestResponseCacheValve.internalProcess(ActiveRequestResponseCacheValve.java:74)
at org.jboss.web.tomcat.service.request.ActiveRequestResponseCacheValve.invoke(ActiveRequestResponseCacheValve.java:47)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:385)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:451)
at java.lang.Thread.run(Thread.java:679)

***************************************************************

We are using the below ,

MySQL 5.5.21
Jboss 5.1.2
OpenJDK 1.6.0_22
Redhat Linux 5.7

I am thinking that the root cause is the connection error that is occurring at the mysql driver level and there is an external change that is causing the communication link failure and this is not originating from the application server. It is manifesting at the driver level. Please confirm?

Could you please let me know the following,

1) Countermeasure to avoid the session disconnection at 3600 seconds?
2) Cause for not performing normally if it takes 3600 seconds ?
3) How to solve this issue ?

Please let me know if you need more information.

Thanks in advance for you suggestions and solutions.

Options: ReplyQuote




Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.