docker-compose部署mysql+canal

一、背景

公司需要给运维的同事制作一套自动化部署的脚本，项目中使用了canal监听数据库做日志处理。

二、需求

通过部署文件能mysql数据库和canal容器搭建，并且canal能成功连接数据库

三、环境

• 操作系统：Linux ubuntu-18.04
• docker：20.10.12
• docker-compose：1.29.2

四、实现步骤

1、制作docker-compose文件

• mysql.yaml

  version: '3.7'
  services:
    mysql-demo:
      # 容器权限与主机共享root权限
      privileged: true
      # 镜像名称
      image: mysql:8.0
      # 容器名称
      container_name: mysql-demo
      # 重启策略
      restart: always
      # 端口映射
      ports:
        - '13306:3306'
      # 卷挂载
      volumes:
        # 本地时间同步
        - /etc/localtime:/etc/localtime
      # 环境变量
      environment:
        MYSQL_ROOT_PASSWORD: 123456
        MYSQL_DATABASE: plm
  

• canal.yaml

  version: '3.7'
  services:
    canal-demo:
      image: canal/canal-server:v1.1.5
      container_name: canal-demo
      privileged: true
      restart: always
      ports:
        - '12111:11111'
      volumes:
        - ./conf/:/home/admin/canal-server/conf/
  

2、部署mysql

• 使用docker-compose启动mysql容器

  # 参数-d表示后台运行
  [email protected]:/home/demo# docker-compose -f mysql.yaml up -d
  Creating network "demo_default" with the default driver
  Creating mysql-demo ... done
  
  # 查看是否启动成功
  [email protected]:/home/demo# docker ps -a
  CONTAINER ID   IMAGE       COMMAND                  CREATED              STATUS              PORTS                                                    NAMES
  a2fc63d02e53   mysql:8.0   "docker-entrypoint.s…"   About a minute ago   Up About a minute   33060/tcp, 0.0.0.0:13306->3306/tcp, :::13306->3306/tcp   mysql-demo
  

• 进入mysql容器，登录root账号

  [email protected]:/home/demo# docker exec -it mysql-demo /bin/bash
  [email protected]:/# mysql -uroot -p123456
  
  # 显示以下信息就登录成功了
  Welcome to the MySQL monitor.  Commands end with ; or g.
  Your MySQL connection id is 8
  Server version: 8.0.27 MySQL Community Server - GPL
  
  Copyright (c) 2000, 2021, Oracle and/or its affiliates.
  
  Oracle is a registered trademark of Oracle Corporation and/or its
  affiliates. Other names may be trademarks of their respective
  owners.
  
  Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
  
  mysql> 
  

• 创建canal账号

  # 依次输入以下命令
  use mysql
  CREATE USER 'canal'@'%' IDENTIFIED BY 'canal';
  GRANT SHOW VIEW, SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%';
  FLUSH PRIVILEGES;
  

• 退出root账号，使用canal账号登录

  mysql> exit
  Bye
  [email protected]:/# mysql -ucanal -pcanal
  
  # 登录成功后输两次exit返回本机
  mysql> exit
  Bye
  [email protected]:/# exit
  exit
  [email protected]:/home/demo#
  

3、部署canal

• 使用docker启动canal容器，将配置文件复制出来

  [email protected]:/home/demo# docker-compose -f canal.yaml up -d
  
  [email protected]:/home/demo# docker run --name canal -d canal/canal-server:v1.1.5
  [email protected]:/home/demo# docker cp canal:/home/admin/canal-server/conf ./conf
  
  # 切换到conf目录下查看配置文件是否复制成功
  [email protected]:/home/demo# cd conf/
  [email protected]:/home/demo/conf# ll
  total 36
  drwxr-xr-x 5 root root 4096 Apr 19  2021 ./
  drwxr-xr-x 3 root root 4096 Dec 23 16:56 ../
  -rwxrwxrwx 1 root root  319 Apr 19  2021 canal_local.properties*
  -rwxrwxrwx 1 root root 6277 Apr 19  2021 canal.properties*
  drwxrwxrwx 2 root root 4096 Dec 23 16:46 example/
  -rwxrwxrwx 1 root root 3437 Apr 19  2021 logback.xml*
  drwxrwxrwx 2 root root 4096 Apr 19  2021 metrics/
  drwxrwxrwx 3 root root 4096 Apr 19  2021 spring/
  

• 配置canal监听信息

  [email protected]:/home/demo/conf# mkdir demo
  [email protected]:/home/demo/conf# cp ./example/instance.properties ./demo/
  [email protected]:/home/demo/conf# cd demo/
  [email protected]:/home/demo/conf/demo# vim instance.properties 
  
  # 将canal.instance.master.address修改为数据库的IP
  canal.instance.master.address=192.168.2.196:13306
  

• 停止canal容器，使用docker-compose部署canal

  # 停止容器
  [email protected]:/home/demo# docker stop canal-demo
  # 删除容器
  [email protected]:/home/demo# docker rm canal-demo
  # 使用docker-compose部署canal
  [email protected]:/home/demo# docker-compose -f canal.yaml up -d
  

• 进入canal容器查看是否成功连接到数据库

  [email protected]:/home/demo# docker exec -it canal-demo /bin/bash
  [[email protected] admin]# cat canal-server/logs/demo/demo.log
  
  # 看到以下信息就成功连接了
  2021-12-23 17:26:07.290 [canal-instance-scan-0] INFO  c.a.otter.canal.instance.spring.CanalInstanceWithSpring - start CannalInstance for 1-demo 
  2021-12-23 17:26:07.306 [canal-instance-scan-0] WARN  c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table filter : ^.*..*$
  2021-12-23 17:26:07.306 [canal-instance-scan-0] WARN  c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table black filter : ^mysql.slave_.*$
  2021-12-23 17:26:07.309 [canal-instance-scan-0] INFO  c.a.otter.canal.instance.core.AbstractCanalInstance - start successful....
  2021-12-23 17:26:07.512 [destination = demo , address = /192.168.2.196:13306 , EventParser] WARN  c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> begin to find start position, it will be long time for reset or first position
  2021-12-23 17:26:07.518 [destination = demo , address = /192.168.2.196:13306 , EventParser] WARN  c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - prepare to find start position just show master status
  2021-12-23 17:26:10.098 [destination = demo , address = /1192.168.2.196:13306 , EventParser] WARN  c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> find start position successfully, EntryPosition[included=false,journalName=binlog.000002,position=4,serverId=1,gtid=<null>,timestamp=1640251442000] cost : 2564ms , the next step is binlog dump
  

五、经历的弯路

最开始是准备通过一个docker-compose文件完成整套部署的，运行之后canal一直连接不上数据库，最后偶然发现canal账号创建之后必须先登录一下，canal才能连的上。