MySQL Forums
Forum List  »  Connector/ODBC

heap error with driver dll
Posted by: Jason Dekok
Date: October 18, 2017 11:52AM

Old system (where it worked for 5 years)
MySQL Server 5.5.13 on windows 2008
ODBC Driver 5.1(5.01.08.00)

New sytem (with issues)
MySQL Server 5.7.12 on Windows 2012
ODBC Driver 5.3.2/5.3.6/5.3.9

I recently moved all my websites from a windows 2008 machine to a new 2012 machine. They connect via odbc to various mysql dbs so I also installed a new version of mysql server on a new 2012 box and transfered my dbs over.

Website code has not changed, mostly classic asp. During testing everything was working fine but after launch when load increased on the web sites we started seeing lots of errors from our top traffic sites. Errors like this:

A process serving application pool 'www.mysite.com' suffered a fatal communication error with the Windows Process Activation Service. The process id was '16952'. The data field contains the error number.

When that happens enough IIS shuts down the app pool. We had microsoft support investigate and they captured some crash dumps and determined it was to do with the odbc dll. See their analysis below. We started with driver version 5.3.6 but tried to downgrade to 5.3.2 which did not work at all (got malformed packet error) so I installed the latest (5.3.9) and it has not made any difference. Any ideas would be greatly appreciated, as you can see in the dump below the statement that caused this error is not complicated and it returns 1 row. Other dumps show the same info with differnt sql statements, none of which are complicated. Both the web and db server are running pretty low cpu and mem usage so they are not being overworked.

Any ideas would be appreciated, at this point my boss wants me to fire up the old servers which I'd really like to avoid doing.


Microsoft Analysis:

I could see the same pattern into all the dumps. Taking just one dump to explain the cause:

DUMP: w3wp__www.mysite.com__pid__10884__date__10_08_2017__time_04_15_09pm__687__second_chance_exception_c0000374.dmp

Analysis:

Callstack of the crashing thread looks like below:

0:094> kL
# ChildEBP RetAddr
00 09c4d310 773a8d58 ntdll!RtlReportCriticalFailure+0x83
01 09c4d320 773a9609 ntdll!RtlpHeapHandleError+0x1c
02 09c4d350 77345fb1 ntdll!RtlpLogHeapFailure+0xa1
03 (Inline) -------- ntdll!RtlpProbeUserBufferUnsafe+0x443ac
04 (Inline) -------- ntdll!RtlpProbeUserBuffer+0x443b6
05 09c4d3ac 6e09ecfa ntdll!RtlFreeHeap+0x443e1
06 09c4d3c0 6e0a4483 msvcr120!free+0x1a
07 09c4d3d8 6e0a4540 msvcr120!__freetlocinfo+0x13e
08 09c4d3ec 6e0a2ccc msvcr120!_updatetlocinfoEx_nolock+0x40
09 09c4d428 6e0a4560 msvcr120!__updatetlocinfo+0x5f
0a 09c4d438 6e0ab90a msvcr120!_LocaleUpdate::_LocaleUpdate+0x3e
0b 09c4d64c 6e0aad30 msvcr120!_input_l+0x10f
0c 09c4d68c 6e0ab727 msvcr120!vscan_fn+0x6a
0d 09c4d6a8 6e1a39da msvcr120!sscanf+0x19
0e 09c4d6dc 6e1a4d46 myodbc5w!SQLNumResultCols+0x2225f
0f 09c4d724 6e197bd3 myodbc5w!SQLNumResultCols+0x235cb
10 09c4d734 6e19757c myodbc5w!SQLNumResultCols+0x16458
11 09c4d75c 6e73e8fb myodbc5w!SQLNumResultCols+0x15e01
12 09c4d7a4 6e740207 odbc32!SetStmtAttr+0x2b6
13 09c4d7d4 6e84c30d odbc32!SQLSetStmtAttrW+0x67
14 (Inline) -------- msdasql!CODBCHandle::SetStmtAttr+0xe
15 (Inline) -------- msdasql!CODBCHandle::OHSetStmtAttr+0x59
16 (Inline) -------- msdasql!CHstmtNode::SetStmtAttr+0x59
17 09c4d828 6e84c0c5 msdasql!CKagRowsetProps::SetODBCOptions+0x1cd
18 09c4d864 6e7fa7c6 msdasql!CKagRowsetProps::FindCapableCC+0x575
19 09c4d944 6e9c4f8b msdasql!CImpICommandText::Execute+0xe36
1a 09c4d9a0 6e9c30b6 msado15!CConnection::Execute+0xcb
1b 09c4dba4 6e9bec15 msado15!_ExecuteAsync+0x20b
1c (Inline) -------- msado15!ExecuteAsync+0x29
1d 09c4dd18 6e9be441 msado15!CQuery::Execute+0x7a1
1e 09c4dd88 6ea0f6d1 msado15!CCommand::_Execute+0x141
1f 09c4de14 6ea08d23 msado15!CConnection::OpenRecordset+0xe1
20 09c4e04c 6ea08627 msado15!CConnection::ExecuteWithModeFlag+0x629
21 09c4e07c 6e9f7134 msado15!CConnection::Execute+0x67
22 09c4e36c 6eba9d6d msado15!CConnection::Invoke+0x30704
23 09c4e3b0 6eba9c9e vbscript!IDispatchInvoke2+0xbf
24 09c4e3e8 6ebb032a vbscript!IDispatchInvoke+0x55
25 09c4e604 6eba92ed vbscript!InvokeDispatch+0x299
26 09c4e62c 6eb83f82 vbscript!InvokeByName+0x48
27 09c4e74c 6eb85eae vbscript!CScriptRuntime::RunNoEH+0x2d0e
28 09c4e79c 6eb85dcb vbscript!CScriptRuntime::Run+0xc3
29 09c4e8ac 6eb98871 vbscript!CScriptEntryPoint::Call+0x10b
2a 09c4e924 6eb98676 vbscript!rtEval+0xdd
2b 09c4e938 6eb85547 vbscript!VbsExecute+0x36
2c 09c4e954 6eb83677 vbscript!StaticEntryPoint::Call+0x2f
2d 09c4ea74 6eb85eae vbscript!CScriptRuntime::RunNoEH+0x2327
2e 09c4eac4 6eb85dcb vbscript!CScriptRuntime::Run+0xc3
2f 09c4ebd4 6eb83677 vbscript!CScriptEntryPoint::Call+0x10


From the callstack it’s pretty clear that the crash happened due to HEAP Corruption.

The Heap corruption happened because of Double FREE à Calling the free( ) on already free block.


************************************************************************************************************************
NT HEAP STATS BELOW
************************************************************************************************************************
**************************************************************
* *
* HEAP ERROR DETECTED *
* *
**************************************************************

Details:

Heap address: 00f50000
Error address: 0d874b78
Error type: HEAP_FAILURE_BLOCK_NOT_BUSY
Details: The caller performed an operation (such as a free or a size check) that is illegal on a free block.



More Info about the DB call being done in this thread was below:

SELECT numParentId FROM tbl_prodcategory_assn WHERE numCategoryId=2410

+0x000 m_mpstrConnectString : CMPString = MySitedb


You are making use of the Oracle component myodbc5w to do the database operation. This seem to be doing Double Free here.

More Info about the component:

0:094> lmvm myodbc5w
Browse full module list
start end module name
6e180000 6e6bc000 myodbc5w C (export symbols) myodbc5w.dll
Loaded symbol image file: myodbc5w.dll
Image path: C:\Program Files (x86)\MySQL\Connector ODBC 5.3\myodbc5w.dll
Image name: myodbc5w.dll
Browse all global symbols functions data
Timestamp: Sat Mar 12 04:09:32 2016 (56E34924)
CheckSum: 00000000
ImageSize: 0053C000
File version: 5.3.6.0
Product version: 5.3.6.0
File flags: 0 (Mask 3)
File OS: 40004 NT Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0409.04e4
CompanyName: Oracle Corporation
ProductName: Connector/ODBC 5.3
InternalName: myodbc5w
OriginalFilename: myodbc5w.dll
ProductVersion: 5, 3, 6, 0
FileVersion: 5, 3, 6, 0
PrivateBuild: Production
SpecialBuild: GA release
FileDescription: MySQL ODBC 5.3 Unicode Driver
LegalCopyright: Copyright (c) 1995, 2013, Oracle and/or its affiliates.
LegalTrademarks: MySQL, MyODBC, Connector/ODBC are trademarks of Oracle Corporation
Comments: provides core driver functionality

Options: ReplyQuote


Subject
Written By
Posted
heap error with driver dll
October 18, 2017 11:52AM
October 26, 2017 09:27AM
October 26, 2017 09:28AM
October 26, 2017 09:46AM
October 26, 2017 09:57AM
November 10, 2017 08:01AM


Sorry, only registered users may post in this forum.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.