使用GoldenGate完成MySQL到MySQL的同步

前端之家收集整理的这篇文章主要介绍了使用GoldenGate完成MySQL到MySQL的同步前端之家小编觉得挺不错的,现在分享给大家,也给大家做个参考。

(一)基础环境配置

  源库 目标库
操作系统版本 CentOS Linux release 7.4 CentOS Linux release 7.4
IP地址 192.168.10.11 192.168.10.12
数据库版本 社区版 5.7.24 社区版 5.7.24
需要同步的数据库 testdb(所有表) testdb_repl
GoldenGate版本 OGG-12.2.0.2 OGG-12.2.0.2

说明:

1.oracle GoldenGate下载地址:https://edelivery.oracle.com/ 免费注册一个账号即可登入,直接搜索“goldengate for non oracle database”,找到想要的版本下载即可。

image_thumb1

2.经过测试,使用OGG-11.2.0.1同步数据库会报错:

ERROR   OGG-00146  VAM function VAMRead returned unexpected result: error 600 - VAM Client Report <CAUSE OF FAILURE : Sanity Check Failed for events
WHEN Failed : While reading log event from binary log
WHERE Failed : MysqLBinLog Reader Module
CONTEXT OF FAILURE : No Information Available!>.

更换为OGG-12.2.0.2后解决,故本文采用的Ogg版本是12.2.0.2。

 

(二)MysqL数据库初始信息

  源库 目标库
MysqL安装位置(basedir) /usr/local/MysqL /usr/local/MysqL
数据存放位置    (datadir) /MysqL/data /MysqL/data
二进制日志位置 (log_bin) /MysqL/binlog /MysqL/binlog

 

(三)MysqL数据库开启二进制日志

要使用ogg抽取数据,必须开启二进制日志。在MysqL配置文件/etc/my.cnf中配置二进制日志,重启数据库生效。

[MysqLd]

# binlog配置
server_id = 1
binlog_format=ROW
log_bin=/MysqL/binlog/master-bin
sync_binlog= 
expire_logs_days=1

 

(四)源端和目标端安装配置GoldenGate(源端和目标端都要执行

(4.1)源端和目标端创建安装ogg,实际上ogg并不需要安装,解压就可以使用

创建ogg安装路径

[root@masterdb ~]# mkdir /ogg

将安装包拷贝到该路径下解压:

[root@masterdb ogg]# ls
ggs_Linux_x64_MysqL_64bit.tar
[root@masterdb ogg]# tar -xvf ggs_Linux_x64_MysqL_64bit. 
./
./MysqL_checklist.sql
...略
./libicuuc.so.56.1
bcpfmt.tpl              emsclnt                        libggnnzitp.so        MysqL.txt
bcrypt.txt              extract                        libggparam.so         notices.txt
cachefiledump           freeBSD.txt                    libggperf.so          oggerr
checkprm                ggcmd                          libggrepo.so          prvtclkm.plb
chkpt_MysqL_create.sql  ggMessage.dat                  libicudata.so.56      replicat
convchk                 ggparam.dat                    libicudata.so.    retrace
convprm                 ggsci                          libicui18n.so.      reverse
db2cntl.tpl             ggs_Linux_x64_MysqL_64bit.tar  libicui18n.so.    server
defgen                  help.txt                       libicuuc.so.        sqlldr.tpl
demo_MysqL_create.sql   keygen                         libicuuc.so.      tcperrs
demo_MysqL_insert.sql   lib                            libMysqLclient.so.18  ucharset.h
demo_MysqL_load.sql     lib12                          libxerces-c-3.1.so    UserExitExamples
demo_MysqL_misc.sql     libantlr3c.so                  logdump               usrdecs.h
dirout                  libdb-6.1.so                   mgr                   zlib.txt
dirwww                  libgglog.so                    MysqL_checklist.sql

 

(4.2)使用ggsci创建ogg相关路径

pwd /ogg [root@masterdb ogg]# ./ggsci Oracle GoldenGate Command Interpreter for MysqL Version 12.2.0.2.2 OGGCORE_12.2.0.2.0_PLATFORMS_170630.0419 Linux,x64,64bit (optimized),MysqL Enterprise on Jun 30 2017 06:32:17 Operating system character set identified as UTF-8. Copyright (C) 1995,2017,Oracle and/or its affiliates. All rights reserved. GGSCI (masterdb) 1> create subdirs Creating subdirectories under current directory /ogg Parameter files /ogg/dirprm: created Report files /ogg/dirrpt: created Checkpoint files /ogg/dirchk: created Process status files /ogg/dirpcs: created sql script files /ogg/dirsql: created Database definitions files /ogg/dirdef: created Extract data files /ogg/dirdat: created Temporary files /ogg/dirtmp: created Credential store files /ogg/dircrd: created Masterkey wallet files /ogg/dirwlt: created Dump files /ogg/dirdmp: created

 

(五)源端配置GoldenGate(源端执行

(5.1)创建数据库用户ogg,该用户用来同步数据

MysqL> GRANT ALL PRIVILEGES ON `testdb`.* TO 'ogg'@%'IDENTIFIED BY ';

 

(5.2)配置管理进程mgr

GGSCI (masterdb)  edit param mgr
port 7809  
dynamicportlist 7840-7939  
purgeoldextracts /ogg/dirdat/*,usecheckpoints,minkeepdays 1 


GGSCI (masterdb) 2> start mgr
Manager started.


GGSCI (masterdb) 3> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING

 

(5.3)配置捕获进程exta

5> edit param exta extract exta sourcedb testdb@192.168.10.11:3306 userid ogg password ogg exttrail /ogg/dirdat/ea discardfile /ogg/dirrpt/exta.dsc,append Tranlogoptions AltLogDest /MysqL/binlog/master-bin.index table testdb.*; GGSCI (masterdb) 10> add extract exta,tranlog,begin now EXTRACT added. GGSCI (masterdb) 11> add exttrail /ogg/dirdat/ea,extract exta EXTTRAIL added.

 

(5.4)配置投递进程dmpa

15> edit param dmpa extract dmpa passthru sourcedb testdb@ userid ogg password ogg rmthost 10.12,mgrport ,compress rmttrail /ogg/dirdat/da dynamicresolution numfiles 3000 table testdb.*16> add extract dmpa,exttrailsource /ogg/dirdat/ea EXTRACT added. GGSCI (masterdb) 17> add rmttrail /ogg/dirdat/da,extract dmpa RMTTRAIL added.

 

(5.5)创建表定义文件,传送到目标端

(5.5.1)创建表定义文件

23> edit param defgen defsfile ./dirdef/testdb.def sourcedb testdb@ userid ogg,password ogg table testdb.*; GGSCI (masterdb) 26> exit

(5.5.2)生成表定义

[root@masterdb ogg]# ./defgen paramfile ./dirprm/defgen.prm  

***********************************************************************
        Oracle GoldenGate Table Definition Generator  MysqL
      Version 
 Linux,1)">07:27:58
 
Copyright (C) or its affiliates. All rights reserved.


                    Starting at 2020-02-08 18:15:40
***********************************************************************

Operating System Version:
Linux
Version #1 SMP Tue Aug 22 21:09:27 UTC 3.10.0-693.el7.x86_64
Node: masterdb
Machine: x86_64
                         soft limit   hard limit
Address Space Size   :    unlimited    unlimited
Heap Size            :    unlimited    unlimited
File Size            :    unlimited    unlimited
cpu Time             :    unlimited    unlimited

Process id: 2510

***********************************************************************
**            Running with the following parameters                  **
***********************************************************************
defsfile ./dirdef/testdb.def
sourcedb testdb@3306 userid ogg,password ***
table testdb.*;
Expanding wildcard table specification testdb.*:

Retrieving definition  testdb.test01.
Retrieving definition  testdb.test02.
Retrieving definition  testdb.test03.


Definitions generated for 3 tables in ./dirdef/testdb.def.

(5.5.3)传送到目标端

scp /ogg/dirdef/testdb.def 10.12:/ogg/dirdef/ root@10.12's password: testdb.def 100% 1785 1.9MB/s 00:00

 

(六)目标端配置GoldenGate(目标端执行

(6.1)创建数据库用户ogg,该用户用来同步数据

ON `testdb_repl`.';

 

(6.2)配置管理进程mgr

GGSCI (slavedb) 3>9> dblogin sourcedb testdb_repl@10.12: userid ogg password ogg 
Successfully logged into database.

GGSCI (slavedb DBLOGIN as ogg)  add checkpointtable testdb_repl.checkpoint  

Successfully created checkpoint table testdb_repl.checkpoint.

GGSCI (slavedb DBLOGIN as ogg) 11> edit params ./GLOBALS 
checkpointtable testdb_repl.checkpoint

 

(6.4)配置应用进程repa

GGSCI (slavedb DBLOGIN as ogg) 13> edit params repa

replicat repa 
targetdb testdb_repl@ userid ogg password ogg  
handlecollisions  
sourcedefs /ogg/dirdef/testdb.def  
discardfile /ogg/dirrpt/repa.dsc,purge  
map testdb.*,target testdb_repl.*;
 

GGSCI (slavedb DBLOGIN as ogg) 16> add replicat repa,exttrail /ogg/dirdat/da,checkpointtable testdb_repl.checkpoint
REPLICAT added.

 

(七)导入初始化数据

将源库需要同步的表导出,然后导入到目标数据库

# 源库导出,并传到目标数据库服务器上 [root@masterdb ~]# MysqLdump -uroot -p123456 -h192.168.10.11 --single-transaction --flush-logs --master-data=2 testdb > testdb.sql MysqLdump: [Warning] Using a password on the command line interface can be insecure. [root@masterdb ~]# anaconda-ks.cfg testdb.sql wgetlog-05-20-22:27.log wgetlog-28.log wget.sh [root@masterdb ~]# scp testdb.sql root@10.12:/root root@s password: testdb.sql 100% 15MB 68.9MB/s 00
# 目标数据库执行导入 [root@slavedb ~]# MysqL -uroot -p123456 testdb_repl < testdb.sql MysqL: [Warning] Using a password on the command line interface can be insecure.

 

(八)开启ogg同步进程

(8.1)源端开启捕获进程(源端执行

由于数据库数据一直在变化,所以对于导出的testdb.sql文件,我们需要记录二进制日志文件的log_file和log_pos

[root@slavedb ~]# cat testdb.sql |grep "CHANGE MASTER TO MASTER_LOG_FILE"
-- CHANGE MASTER TO MASTER_LOG_FILE=master-bin.000005',MASTER_LOG_POS=154;

然后从导出时的binlog开始抽取

5> alter extract exta,vam,lognum 5,logpos 154 EXTRACT altered. GGSCI (masterdb) 6> start exta Sending START request to MANAGER ... EXTRACT EXTA starting

 

(8.2)开启投递进程(源端执行

19> start dmpa Sending START request to MANAGER ... EXTRACT DMPA starting

 

(8.3)开启应用进程(目标端执行

GGSCI (slavedb DBLOGIN as ogg) 18> start repa

Sending START request to MANAGER ...
REPLICAT REPA starting

 

(九)确认同步状态

(9.1)使用info all查看进程是否运行正常

所有进程状态为“running”代表正常

源端:

8> info all Program Status Group Lag at Chkpt Time Since Chkpt MANAGER RUNNING EXTRACT RUNNING DMPA 00 08 EXTRACT RUNNING EXTA 04

 

目标端:

23> all Program Status Group Lag at Chkpt Time Since Chkpt MANAGER RUNNING REPLICAT RUNNING REPA 01:14:40 00

 

(9.2)源端手动插入数据,查看目标端是否同步

源端执行:

insert into test03 values(1,1)">a');

目标端查看数据是否同步过来:

MysqL> use testdb_repl ;
Database changed

MysqL> select * from test03;
+-----+------+
| id3 | name |
+-----+------+
|   1 | a    |
+-----+------+
1 row in set (0.04 sec)

 

 

【完】

猜你在找的MySQL相关文章