Friday, May 02, 2008

How to mirror the MediaWiki @ Windows Platform.

Ready to do:
1.Apache
2.Mysql
3.php
4.MediaWiki
5.download wiki*.xml.bz from http://download.wikimedia.org/ and extract .xml from .bz
6.ActivePerl
7.download mwimport from http://meta.wikimedia.org/wiki/Data_dumps/mwimport
or
8.Java
9.download mwdumper from http://www.mediawiki.org/wiki/MWDumper

Run at command line:
type wiki-.xml perl mwimport.pl mysql -f -u -p
or


Tips:
1.mysql's Path should in PATH enviourment variable, otherwise the command cannot execute correctly.
2.modify fields' size in the table.sql on installing MediaWiki, otherwise there are some error.
page_touched binary(32) NOT NULL default '',
rev_comment blob NOT NULL,

wiki is cool, Damn GFW.
-------------------------------
That is my mirror's result.



F:\>type zhwiki-pages-articles.xml perl mwimport.pl mysql -f -u root -p wiki
db
Enter password: siteinfo: untested generator 'MediaWiki 1.13alpha', expect troub
le ahead
****
1000 pages (125.000/s), 1000 revisions (125.000/s) in 8 seconds
2000 pages (153.846/s), 2000 revisions (153.846/s) in 13 seconds
3000 pages (176.471/s), 3000 revisions (176.471/s) in 17 seconds
4000 pages (181.818/s), 4000 revisions (181.818/s) in 22 seconds
5000 pages (192.308/s), 5000 revisions (192.308/s) in 26 seconds
6000 pages (187.500/s), 6000 revisions (187.500/s) in 32 seconds
7000 pages (179.487/s), 7000 revisions (179.487/s) in 39 seconds
8000 pages (190.476/s), 8000 revisions (190.476/s) in 42 seconds
ERROR 1366 (HY000) at line 25060: Incorrect string value: '\xF0\xAA\x95\xA8' for
column 'page_title' at row 124
9000 pages (163.636/s), 9000 revisions (163.636/s) in 55 seconds
10000 pages (153.846/s), 10000 revisions (153.846/s) in 65 seconds
11000 pages (144.737/s), 11000 revisions (144.737/s) in 76 seconds
12000 pages (136.364/s), 12000 revisions (136.364/s) in 88 seconds
13000 pages (127.451/s), 13000 revisions (127.451/s) in 102 seconds
14000 pages (134.615/s), 14000 revisions (134.615/s) in 104 seconds
15000 pages (123.967/s), 15000 revisions (123.967/s) in 121 seconds
16000 pages (117.647/s), 16000 revisions (117.647/s) in 136 seconds
17000 pages (112.583/s), 17000 revisions (112.583/s) in 151 seconds
18000 pages (117.647/s), 18000 revisions (117.647/s) in 153 seconds
ERROR 1366 (HY000) at line 51865: Incorrect string value: '\xF0\xA8\xA7\x80' for
column 'page_title' at row 986
19000 pages (112.426/s), 19000 revisions (112.426/s) in 169 seconds
20000 pages (108.108/s), 20000 revisions (108.108/s) in 185 seconds
21000 pages (105.000/s), 21000 revisions (105.000/s) in 200 seconds
22000 pages (102.804/s), 22000 revisions (102.804/s) in 214 seconds
23000 pages (106.481/s), 23000 revisions (106.481/s) in 216 seconds
24000 pages (103.004/s), 24000 revisions (103.004/s) in 233 seconds
25000 pages (106.383/s), 25000 revisions (106.383/s) in 235 seconds
26000 pages (102.767/s), 26000 revisions (102.767/s) in 253 seconds
27000 pages (105.469/s), 27000 revisions (105.469/s) in 256 seconds
ERROR 1366 (HY000) at line 77942: Incorrect string value: '\xF0\xA8\xA8\x8F' for
column 'page_title' at row 845
28000 pages (102.190/s), 28000 revisions (102.190/s) in 274 seconds
29000 pages (105.072/s), 29000 revisions (105.072/s) in 276 seconds
30000 pages (108.303/s), 30000 revisions (108.303/s) in 277 seconds
31000 pages (102.990/s), 31000 revisions (102.990/s) in 301 seconds
32000 pages (105.611/s), 32000 revisions (105.611/s) in 303 seconds
33000 pages (101.538/s), 33000 revisions (101.538/s) in 325 seconds
34000 pages (103.976/s), 34000 revisions (103.976/s) in 327 seconds
35000 pages (100.575/s), 35000 revisions (100.575/s) in 348 seconds
36000 pages (102.857/s), 36000 revisions (102.857/s) in 350 seconds
37000 pages ( 98.667/s), 37000 revisions ( 98.667/s) in 375 seconds
38000 pages (100.796/s), 38000 revisions (100.796/s) in 377 seconds
39000 pages ( 97.744/s), 39000 revisions ( 97.744/s) in 399 seconds
40000 pages ( 99.751/s), 40000 revisions ( 99.751/s) in 401 seconds
41000 pages ( 96.698/s), 41000 revisions ( 96.698/s) in 424 seconds
42000 pages ( 98.824/s), 42000 revisions ( 98.824/s) in 425 seconds
43000 pages ( 95.556/s), 43000 revisions ( 95.556/s) in 450 seconds
44000 pages ( 97.561/s), 44000 revisions ( 97.561/s) in 451 seconds
45000 pages ( 94.142/s), 45000 revisions ( 94.142/s) in 478 seconds
46000 pages ( 96.033/s), 46000 revisions ( 96.033/s) in 479 seconds
47000 pages ( 93.439/s), 47000 revisions ( 93.439/s) in 503 seconds
48000 pages ( 95.238/s), 48000 revisions ( 95.238/s) in 504 seconds
49000 pages ( 92.453/s), 49000 revisions ( 92.453/s) in 530 seconds
50000 pages ( 93.985/s), 50000 revisions ( 93.985/s) in 532 seconds
51000 pages ( 91.071/s), 51000 revisions ( 91.071/s) in 560 seconds
52000 pages ( 92.527/s), 52000 revisions ( 92.527/s) in 562 seconds
53000 pages ( 89.983/s), 53000 revisions ( 89.983/s) in 589 seconds
54000 pages ( 91.525/s), 54000 revisions ( 91.525/s) in 590 seconds
55000 pages ( 88.997/s), 55000 revisions ( 88.997/s) in 618 seconds
56000 pages ( 90.323/s), 56000 revisions ( 90.323/s) in 620 seconds
ERROR 1406 (22001) at line 161918: Data too long for column 'rev_comment' at row
1861
57000 pages ( 88.099/s), 57000 revisions ( 88.099/s) in 647 seconds
58000 pages ( 89.368/s), 58000 revisions ( 89.368/s) in 649 seconds
59000 pages ( 87.407/s), 59000 revisions ( 87.407/s) in 675 seconds
60000 pages ( 88.626/s), 60000 revisions ( 88.626/s) in 677 seconds
61000 pages ( 87.019/s), 61000 revisions ( 87.019/s) in 701 seconds
62000 pages ( 88.193/s), 62000 revisions ( 88.193/s) in 703 seconds
ERROR 1366 (HY000) at line 182887: Incorrect string value: '\xF0\xA8\xAE\x81' fo
r column 'page_title' at row 1339
63000 pages ( 86.420/s), 63000 revisions ( 86.420/s) in 729 seconds
64000 pages ( 84.993/s), 64000 revisions ( 84.993/s) in 753 seconds
65000 pages ( 86.207/s), 65000 revisions ( 86.207/s) in 754 seconds
66000 pages ( 87.302/s), 66000 revisions ( 87.302/s) in 756 seconds
67000 pages ( 88.507/s), 67000 revisions ( 88.507/s) in 757 seconds
68000 pages ( 86.735/s), 68000 revisions ( 86.735/s) in 784 seconds
69000 pages ( 87.786/s), 69000 revisions ( 87.786/s) in 786 seconds
70000 pages ( 86.101/s), 70000 revisions ( 86.101/s) in 813 seconds
71000 pages ( 87.117/s), 71000 revisions ( 87.117/s) in 815 seconds

...

F:\>java -jar mwdumper.jar --format=sql:1.5 zhwiki-pages-articles.xml mysql -f
-u root -p wikidb
Enter password: ****
1,000 pages (262.329/sec), 1,000 revs (262.329/sec)
2,000 pages (329.924/sec), 2,000 revs (329.924/sec)
3,000 pages (381.728/sec), 3,000 revs (381.728/sec)
4,000 pages (399.401/sec), 4,000 revs (399.401/sec)
5,000 pages (391.696/sec), 5,000 revs (391.696/sec)
6,000 pages (308.928/sec), 6,000 revs (308.928/sec)
7,000 pages (290.155/sec), 7,000 revs (290.155/sec)
8,000 pages (293.244/sec), 8,000 revs (293.244/sec)
9,000 pages (264.955/sec), 9,000 revs (264.955/sec)
ERROR 1366 (HY000) at line 122: Incorrect string value: '\xF0\xAA\x95\xA8' for c
olumn 'page_title' at row 167
10,000 pages (238.989/sec), 10,000 revs (238.989/sec)
11,000 pages (204.594/sec), 11,000 revs (204.594/sec)
12,000 pages (186.408/sec), 12,000 revs (186.408/sec)
13,000 pages (173.082/sec), 13,000 revs (173.082/sec)