Migration failed at Bulk Data Transfer

[yao zhang]

I am trying to migrate a schema within the same MySQL server (i.e. like copying tables within the same MySQL server), but I always failed at Bulk Data Transfer.
This error also occurs while I tried to migrate data from MS SQL server to MySQL server. I already tried to change max_allowed_packet=0.3G and only allow 1 worker taskes. I hope anyone can help me out because I will get many MS SQL data and migrate them to MySQL server (on Ubuntu).

MySQL server version is 5.0.67-community-nt;
MySQL workbench version is 5.2.41 CE;
My OS is Windows XP Professional Version 2002 with SP3.
My Available RAM for MySQL<1G.
Here is the log:


Starting...
Prepare information for data copy...
Prepare information for data copy done
Determine number of rows to copy....
Counting number of rows in tables...

wbcopytables.exe --count-only --passwords-from-stdin --mysql-source=yzhang@localhost:3306 --table `hhp3_raw` `members` --table `hhp3_raw` `dihy3` --table `hhp3_raw` `dihy2` --table `hhp3_raw` `lcount` --table `hhp3_raw` `claims` --table `hhp3_raw` `target` --table `hhp3_raw` `dcount` --table `hhp3_raw` `pg` --table `hhp3_raw` `newdata` --table `hhp3_raw` `pcong`

18:01:34 [INF][ copytable]: Connecting to MySQL server at localhost:3306 with user yzhang
18:01:34 [INF][ copytable]: Connection to MySQL opened

8467517 total rows in 10 tables need to be copied:
- `hhp3_raw`.`members`: 113000
- `hhp3_raw`.`dihy3`: 71435
- `hhp3_raw`.`dihy2`: 76038
- `hhp3_raw`.`lcount`: 361484
- `hhp3_raw`.`claims`: 2668990
- `hhp3_raw`.`target`: 70942
- `hhp3_raw`.`dcount`: 818241
- `hhp3_raw`.`pg`: 17
- `hhp3_raw`.`newdata`: 4287325
- `hhp3_raw`.`pcong`: 45

Determine number of rows to copy finished
Copy data to target RDBMS....

Migrating data...

wbcopytables.exe --mysql-source=yzhang@localhost:3306 --target=yzhang@localhost:3306 --progress --passwords-from-stdin --truncate-target --thread-count=1 --table `hhp3_raw` `members` `tt` `members` --table `hhp3_raw` `dihy3` `tt` `dihy3` --table `hhp3_raw` `dihy2` `tt` `dihy2` --table `hhp3_raw` `lcount` `tt` `lcount` --table `hhp3_raw` `claims` `tt` `claims` --table `hhp3_raw` `target` `tt` `target` --table `hhp3_raw` `dcount` `tt` `dcount` --table `hhp3_raw` `pg` `tt` `pg` --table `hhp3_raw` `newdata` `tt` `newdata` --table `hhp3_raw` `pcong` `tt` `pcong`

`tt`.`members`:Copying 4 columns of 113000 rows from table `hhp3_raw`.`members`
*** ERROR: `tt`.`members`:Not enough memory to allocate insert buffer of size 1073741824
`tt`.`members`:Finished copying 0 rows in 0m01s
`tt`.`dihy3`:Copying 4 columns of 71435 rows from table `hhp3_raw`.`dihy3`
*** ERROR: `tt`.`dihy3`:Not enough memory to allocate insert buffer of size 1073741824
`tt`.`dihy3`:Finished copying 0 rows in 0m00s
`tt`.`dihy2`:Copying 4 columns of 76038 rows from table `hhp3_raw`.`dihy2`
*** ERROR: `tt`.`dihy2`:Not enough memory to allocate insert buffer of size 1073741824
`tt`.`dihy2`:Finished copying 0 rows in 0m00s
`tt`.`lcount`:Copying 5 columns of 361484 rows from table `hhp3_raw`.`lcount`
*** ERROR: `tt`.`lcount`:Not enough memory to allocate insert buffer of size 1073741824
`tt`.`lcount`:Finished copying 0 rows in 0m02s
`tt`.`claims`:Copying 15 columns of 2668990 rows from table `hhp3_raw`.`claims`
*** ERROR: `tt`.`claims`:Not enough memory to allocate insert buffer of size 1073741824
`tt`.`claims`:Finished copying 0 rows in 0m33s
`tt`.`target`:Copying 4 columns of 70942 rows from table `hhp3_raw`.`target`
*** ERROR: `tt`.`target`:Not enough memory to allocate insert buffer of size 1073741824
`tt`.`target`:Finished copying 0 rows in 0m00s
`tt`.`dcount`:Copying 5 columns of 818241 rows from table `hhp3_raw`.`dcount`
*** ERROR: `tt`.`dcount`:Not enough memory to allocate insert buffer of size 1073741824
`tt`.`dcount`:Finished copying 0 rows in 0m04s
`tt`.`pg`:Copying 3 columns of 17 rows from table `hhp3_raw`.`pg`
*** ERROR: `tt`.`pg`:Not enough memory to allocate insert buffer of size 1073741824
`tt`.`pg`:Finished copying 0 rows in 0m00s
`tt`.`newdata`:Copying 26 columns of 4287325 rows from table `hhp3_raw`.`newdata`
*** ERROR: `tt`.`newdata`:Not enough memory to allocate insert buffer of size 1073741824
`tt`.`newdata`:Finished copying 0 rows in 3m10s
`tt`.`pcong`:Copying 3 columns of 45 rows from table `hhp3_raw`.`pcong`
*** ERROR: `tt`.`pcong`:Not enough memory to allocate insert buffer of size 1073741824
`tt`.`pcong`:Finished copying 0 rows in 0m00s
Copy helper has finished

Data copy results:
- `tt`.`members` has FAILED (0 of 113000 rows copied)
- `tt`.`dihy3` has FAILED (0 of 71435 rows copied)
- `tt`.`dihy2` has FAILED (0 of 76038 rows copied)
- `tt`.`lcount` has FAILED (0 of 361484 rows copied)
- `tt`.`claims` has FAILED (0 of 2668990 rows copied)
- `tt`.`target` has FAILED (0 of 70942 rows copied)
- `tt`.`dcount` has FAILED (0 of 818241 rows copied)
- `tt`.`pg` has FAILED (0 of 17 rows copied)
- `tt`.`newdata` has FAILED (0 of 4287325 rows copied)
- `tt`.`pcong` has FAILED (0 of 45 rows copied)
0 tables of 10 were fully copied
Copy data to target RDBMS finished

Tasks finished with warnings and/or errors, view the logs for details

Finished performing tasks.



Edited 1 time(s). Last edit at 08/14/2012 08:51AM by yao zhang.