MySQL Forums
Forum List  »  PHP

Re: Concat Occasionally Drops a Character or Two
Posted by: Jonathan Shaltz
Date: September 27, 2009 03:03PM

Sure thing, and thanks for helping!

It appears that the problem occurs equally for INSERTs and UPDATEs: pushing each chunk into the database as its own BLOB, some of them end up with a length of 99,999 rather than 100,000. Sure enough, these same chunks increment LENGTH(Content) by 99,999 bytes rather than 100,000. I verified again that the length of each chunk in PHP's array is exactly 100,000 bytes.

I had added the BINARY modifier recently, but it had no effect. I removed it for this test.

This is my original table:

CREATE TABLE `media` (
 `Media` int(11) NOT NULL AUTO_INCREMENT,
 `User` int(11) NOT NULL,
 `Type` int(11) NOT NULL,
 `Name` varchar(100) NOT NULL,
 `Content` longblob NOT NULL,
 `Thumbnail` blob,
 PRIMARY KEY (`Media`),
 KEY `User` (`User`),
 KEY `Type` (`Type`),
 CONSTRAINT `media_ibfk_1` FOREIGN KEY (`User`) REFERENCES `users` (`User`) ON DELETE CASCADE,
 CONSTRAINT `media_ibfk_2` FOREIGN KEY (`Type`) REFERENCES `mediatypes` (`Type`) ON DELETE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=189 DEFAULT CHARSET=latin1


Just in case you meant the new test table:

CREATE TABLE `blobtest` (
 `PK` int(11) NOT NULL AUTO_INCREMENT,
 `Name` varchar(50) NOT NULL,
 `Data` longblob NOT NULL,
 PRIMARY KEY (`PK`)
) ENGINE=InnoDB AUTO_INCREMENT=139 DEFAULT CHARSET=latin1


This is the log output with a relatively large file. Length is calculated with LENGTH().

Size = 2,847,124 -- The original file size as reported by PHP with filesize()
Len(Chunk #1) = 100,000 -- The size of a blob created for testing purposes in a new table, created as you suggested
Len(Chunk #2) = 100,000
Len(Chunk #3) = 100,000
Len(Chunk #4) = 100,000
Len(Chunk #5) = 99,999 -- Chunk #5 is the first to omit a char
Len(Chunk #6) = 99,999
Len(Chunk #7) = 100,000
Len(Chunk #8) = 100,000
Len(Chunk #9) = 100,000
Len(Chunk #10) = 100,000
Len(Chunk #11) = 100,000
Len(Chunk #12) = 99,999
Len(Chunk #13) = 100,000
Len(Chunk #14) = 100,000
Len(Chunk #15) = 100,000
Len(Chunk #16) = 100,000
Len(Chunk #17) = 100,000
Len(Chunk #18) = 100,000
Len(Chunk #19) = 100,000
Len(Chunk #20) = 100,000
Len(Chunk #21) = 100,000
Len(Chunk #22) = 99,997
Len(Chunk #23) = 99,999
Len(Chunk #24) = 100,000
Len(Chunk #25) = 100,000
Len(Chunk #26) = 100,000
Len(Chunk #27) = 100,000
Len(Chunk #28) = 100,000
Len(Chunk #29) = 47,123 -- Final chunk
After first chunk, len = 100,000 -- "Seed" is OK
After concat, len = 200,000 -- First concat (second chunk) is OK
After concat, len = 300,000 -- Chunk #3
After concat, len = 400,000 -- Chunk #4
After concat, len = 499,999 -- There's that damaged fifth chunk
After concat, len = 599,998
After concat, len = 699,998
After concat, len = 799,998
After concat, len = 899,998
After concat, len = 999,998
After concat, len = 1,099,998
After concat, len = 1,199,997
After concat, len = 1,299,997
After concat, len = 1,399,997
After concat, len = 1,499,997
After concat, len = 1,599,997
After concat, len = 1,699,997
After concat, len = 1,799,997
After concat, len = 1,899,997
After concat, len = 1,999,997
After concat, len = 2,099,997
After concat, len = 2,199,994
After concat, len = 2,299,993
After concat, len = 2,399,993
After concat, len = 2,499,993
After concat, len = 2,599,993
After concat, len = 2,699,993
After concat, len = 2,799,993
After concat, len = 2,847,116 -- Only missing eight bytes out of 2.8 MB, but that's enough to corrupt a file

This is the PHP code, with the new scaffolding in place:

$chunk = file_get_contents($path, FILE_BINARY, null, ($chunkNum-1) * MAX_CHUNK_SIZE, MAX_CHUNK_SIZE);
logToFile("Chunk #$chunkNum is " . numToText(strlen($chunk)) . " bytes");  // Reports 100,000
$content[$chunkNum] = self::cleanForSQL($chunk);
$sql = "INSERT INTO BlobTest (Name, Data) VALUES ('$chunkNum', '" . $content[$chunkNum] . "')";
self::$db->query_action($sql);
logToFile("Len(Chunk #$chunkNum) = " . numToText(self::$db->query_scalar("SELECT LENGTH(Data) FROM BlobTest WHERE PK = (SELECT MAX(PK) FROM BlobTest)")));  // May report 99,999

Options: ReplyQuote




Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.