一. 环境描述
操作系统：RHEL 6.4
mysql ：5.6.27
|
角色 |
IP 地址 |
主机名 |
Server ID |
类型 |
|
主库 |
192.168.222.156 |
oradb |
10 |
写入 |
|
备库、MHA管理服务器 |
192.168.222.157 |
mysqldb2 |
11 |
读、监控集群组 |
二、部署MHA
1.主库安装 MHA NODE
1.1 安装需要的PERL模块
yum -y install *zlib-dev*
install.sh
#!/bin/bash
wget http://xrl.us/cpanm
–no-check-certificate
mv cpanm /usr/bin
chmod 755 /usr/bin/cpanm
cat >/root/list<<EOF
install DBD::mysql
EOF
for package in `cat /root/list`
do
cpanm $package
done
1.2 安装 MHA NODE
从 https://github.com/yoshinorim/mha4mysql-node 下载最新版本的 mha4mysql-node 源代码（目前是 0.57版）
unzip mha4mysql-node-master.zip
cd mha4mysql-node-master
perl Makefile.PL
make
make install
2. 备库安装 MHA Manager
2.1 安装需要的perl模块
yum -y install *zlib-dev*
install.sh
#!/bin/bash
wget http://xrl.us/cpanm
–no-check-certificate
mv cpanm /usr/bin
chmod 755 /usr/bin/cpanm
cat >/root/list<<EOF
install DBD::mysql
install Config::Tiny
install Log::Dispatch
install Parallel::ForkManager
install Time::HiRes
EOF
for package in `cat /root/list`
do
cpanm $package
done
2.2 安装MHA NODE
从 https://github.com/yoshinorim/mha4mysql-node 下载最新版本的 mha4mysql-node 源代码（目前是 0.57版）
unzip mha4mysql-node-master.zip
cd mha4mysql-node-master
perl Makefile.PL
make
make install
2.3 安装 MHA Manager
从 https://github.com/yoshinorim/mha4mysql-manager下载最新版本的 mha4mysql-manager 源代码（目前是 0.57版）
unzip mha4mysql-manager-master.zip
cd mha4mysql-manager-master
perl Makefile.PL
make
make install
3. 配置 SSH 登录无密码验证
3.1 主库配置到 MHA manager 节点（备库节点）SSH登录无密码验证
vi /etc/hosts
192.168.222.157 mysqldb2
192.168.222.156 oradb
ssh-keygen -t rsa
ssh-coyp-id -i /root/.ssh/id_rsa.pub root@192.168.222.157
ssh-coyp-id -i /root/.ssh/id_rsa.pub root@192.168.222.156
ssh root@192.168.222.156
ssh root@oradb
ssh root@192.168.222.157
ssh root@mysqldb2
3.2 MHA manager 节点（备库节点）配置到主库配置 SSH登录无密码验证
vi /etc/hosts
192.168.222.157 mysqldb2
192.168.222.156 oradb
ssh-keygen -t rsa
ssh-coyp-id -i /root/.ssh/id_rsa.pub root@192.168.222.157
ssh-coyp-id -i /root/.ssh/id_rsa.pub root@192.168.222.156
ssh root@192.168.222.156
ssh root@oradb
ssh root@192.168.222.157
ssh root@mysqldb2
4. 搭建主从复制
4.1 修改主库my.cnf 配置服务器ID为10
vi /app/oracle/oradata2/mysql-5.6.27/my.cnf
[mysqld]
server_id = 10
4.1 修改备库my.cnf 配置
vi /usr/local/mysql/my.cnf
[mysqld]
server_id = 10
# slave config
log_bin=/usr/local/mysql/data/mysql-bin
relay_log=/usr/local/mysql/data/relay-bin
binlog_format=row
relay_log_purge=0
read_only=1
4.2 在主库上备份
mysqldump –master-data=2 –single-transaction -R -A
> all.sql
其中 –master-data=2代表备份时刻记录 master的 Binlog 位置和 Position，–single-transaction 意思是获取一致性快照， -R 意思是备份相关的存储过程
，-A代表备份所有的 schema。
4.3 在主库上创建复制用户和 MHA监控用户
mysql> grant replication slave on *.* to ‘repl’@’192.168.222.%’ identified by ‘123456’;
mysql> grant all on *.* to ‘root’@’192.168.222.%’ identified by ‘123456’; /*MHA 监控用户*/
4.4 查看主库上备份时刻 Binlog 的名称和位置，MASTER_LOG和 MASTER_LOG_POS
head -n 30 all.sql|grep -i
“CHANGE MASTER TO”
— CHANGE MASTER TO
MASTER_LOG_FILE=’mysql-bin.000068′, MASTER_LOG_POS=325;
4.5 将备份复制到备库
scp all.sql root@192.168.222.157:/root/olm
4.6 在备库导入数据
mysql> source /root/olm/all.sql
mysql> flush privileges;
mysql>
change master to
master_host=’192.168.222.156′,master_user=’repl’,master_password=’123456′,
master_port=3306,master_log_file=’mysql-bin.000068′,master_log_pos=325;
start slave;
—查看复制状态
show slave status \G
5. 配置MHA
5.1 创建 MHA 工作目录，并且创建相关配置文件
mkdir -p /etc/masterha
vi /etc/masterha/app1.cnf
[server default]
manager_log=/masterha/app1/app1.log
manager_workdir=/masterha/app1
master_binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data
master_ip_failover_script=/usr/local/bin/master_ip_failover
master_ip_online_change_script=/usr/local/bin/master_ip_online_change
password=123456
ping_interval=1
remote_workdir=/tmp
repl_password=123456
repl_user=repl
secondary_check_script=/usr/local/bin/masterha_secondary_check
-s mysqldb2 -s oradb –user=root –master_host=oradb
–master_ip=’192.168.222.156′ –master_port=3306
shutdown_script=””
ssh_user=root
user=root
[server1]
hostname=192.168.222.156
port=3306
[server2]
candidate_master=1
check_repl_delay=0
hostname=192.168.222.157
port=3306
5.2 设置定期清理中继日志的脚本
crontab -e
0 4 * * *
/usr/local/bin/purge_relay_logs –user=root –password=123456 –disable_replay_log_purge –port=3306 –workdir=/usr/local/mysql/data –disable_relay_log_purge >> /usr/loca/masterha/log/purge_relay_logs.log 2>&1
5.2 编辑 /etc/bashrc 文件，在文件末尾添加以下内容，避免MHA执行故障切换时无法找到 mysqlbinlog
PATH=”$PATH:/usr/local/mysql/bin”
export PATH
5.3 检查 SSH 的配置
masterha_check_ssh –conf=/etc/masterha/app1.cnf
5.4 检查MYSQL复制
masterha_check_repl –conf=/etc/masterha/app1.cnf
5.5 启动 MHA Manger 监控
tail -f /masterha/app1/app1.log
—在另一个SSH 窗口启动 MHA Manger 监控。–remove_dead_master_conf 该参数代表当发生主从切换后，老的主库的IP将会从配置文件中移除。
–ignore_last_failover 在缺省情况下，如果MHA检测到连续发生宕机，而且两次宕机时间不足8小时的话，则不会进行 Failover，之所以这样限制
—是为了避免 ping-pong效应。该参数代表忽悠上次 MHA 触发切换产生的文件，默认情况下，MHA 发生切换后将会在 /masterha/paa1下产生 app1.
–failover.complete 文件，下次再次切换的时候如果发现目录存在该文件将不允许触发切换，除非在第一次切换后收到 rm -rf /masterha/paa1/app1.
–failover.complete,出于方便考虑，我们每次启动MHA时会添加 –ignore_last_failover 参数。
nohup masterha_manager
–conf=/etc/masterha/app1.cnf –remove_dead_master_conf
–ignore_last_failover< /dev/null >/masterha/app1/manager.log 2>&1
&
—查看 MHA Manager 监控是否正常
masterha_check_status –conf=/etc/masterha/app1.cnf
app1 (pid:4758) is
running(0:PING_OK), master:192.168.222.156
—在默认情况下，10秒内状态为10：INITIALIZING_MONITOR，当状态转变为0:PING_OK后表明已经开启了到 MASTER 端的监控。
关闭MHA的命令如下：
masterha_stop –conf=/etc/masterha/app1.cnf
6. 配置 VIP
6.1 在主库下载集群心跳软件 keepalived，并进行安装
访问 http://www.keepalived.org/software/ 下载 keepalived-1.2.18.tar.gz
tar vzxf keepalived-1.2.18.tar.gz
cd keepalived-1.2.18
./configure
–prefix=/usr/local/keepalived
make
make install
cp
/usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
cp
/usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
cp
/usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp
/usr/local/keepalived/sbin/keepalived /usr/sbin/
cd /etc/keepalived/
vi keepalived.conf
—下面是添加的配置
[root@oradb ~]# cat
/etc/keepalived/keepalived.conf
! Configuration File for
keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id MySQL-ha
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.222.158/24
}
}
service keepalived start
—是用 ip add 命令可以看到 eth0上已经绑定了虚拟IP 192.168.222.158 ，注意使用 ifconfig 是看不到虚拟IP的
[root@oradb ~]# ip add
1: lo:
<LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP>
mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:53:7e:f0 brd ff:ff:ff:ff:ff:ff
inet 192.168.222.156/24 brd 192.168.222.255 scope global eth0
inet 192.168.222.158/24
scope global secondary eth0
inet6 fe80::20c:29ff:fe53:7ef0/64 scope link
valid_lft forever preferred_lft forever
3: eth2:
<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen
1000
link/ether 00:0c:29:53:7e:fa brd ff:ff:ff:ff:ff:ff
inet 192.168.1.11/24 brd 192.168.1.255 scope global eth2
inet6 fe80::20c:29ff:fe53:7efa/64 scope link
valid_lft forever preferred_lft forever
6.2 在备库下载集群心跳软件 keepalived，并进行安装
访问 http://www.keepalived.org/software/ 下载 keepalived-1.2.18.tar.gz
tar vzxf keepalived-1.2.18.tar.gz
cd keepalived-1.2.18
./configure
–prefix=/usr/local/keepalived
make
make install
cp
/usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
cp
/usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
cp
/usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp
/usr/local/keepalived/sbin/keepalived /usr/sbin/
cd /etc/keepalived/
vi keepalived.conf
—下面是添加的配置
! Configuration File for
keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id MySQL-ha
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 120
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.222.158/24
}
}
service keepalived start
6.2 MHA引入 keepalived
在MHA监控节点（备库）编辑 master_ip_failover 脚本
vi /usr/local/bin/master_ip_failover
#!/usr/bin/env perl
# Copyright (C) 2011 DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
## Note: This is a sample script
and is not complete. Modify the script based on your environment.
use strict;
use warnings FATAL => ‘all’;
use Getopt::Long;
use MHA::DBHelper;
my (
$command, $ssh_user, $orig_master_host,
$orig_master_ip, $orig_master_port, $new_master_host,
$new_master_ip, $new_master_port, $new_master_user,
$new_master_password, $cmd
);
GetOptions(
‘command=s’ =>
\$command,
‘ssh_user=s’ =>
\$ssh_user,
‘orig_master_host=s’ =>
\$orig_master_host,
‘orig_master_ip=s’ =>
\$orig_master_ip,
‘orig_master_port=i’ =>
\$orig_master_port,
‘new_master_host=s’ => \$new_master_host,
‘new_master_ip=s’ =>
\$new_master_ip,
‘new_master_port=i’ =>
\$new_master_port,
‘new_master_user=s’ =>
\$new_master_user,
‘new_master_password=s’ => \$new_master_password,
);
exit &main();
sub main {
if ( $command eq “stop” || $command eq “stopssh” ) {
# $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = 1;
eval {
# updating global catalog, etc
$exit_code = 0;
};
if ($@) {
warn “Got Error: $@\n”;
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq “start” ) {
# all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=0, etc)
here.
my $exit_code = 10;
eval {
my $new_master_handler = new
MHA::DBHelper();
# args: hostname, port, user, password,
raise_error_or_not
$new_master_handler->connect(
$new_master_ip, $new_master_port,
$new_master_user, $new_master_password,
1 );
## Set read_only=0 on the new master
$new_master_handler->disable_log_bin_local();
print “Set read_only=0 on the new
master.\n”;
$new_master_handler->disable_read_only();
## Creating an app user on the new master
print “Creating app user on the new
master..\n”;
#FIXME_xxx_create_user(
$new_master_handler->{dbh} );
#$new_master_handler->enable_log_bin_local();
#$new_master_handler->disconnect();
## Update master ip on the catalog
database, etc
#FIXME_xxx;
#$cmd = ‘ssh root@192.168.222.156 \./lvs-admin stop’;
#$cmd =
‘ssh’.$ssh_user.’@’.$orig_master_ip.’\’./lvs-admin stop\”;
$cmd = ‘ssh ‘.$ssh_user.’@’.$orig_master_ip.’ \./lvs-admin stop’;
system($cmd);
$exit_code = 0;
};
if ($@) {
warn $@;
# If you want to continue failover, exit
10.
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq “status” ) {
# do nothing
exit 0;
}
else {
&usage();
exit 1;
}
}
sub usage {
“Usage: master_ip_failover –command=start|stop|stopssh|status
–orig_master_host=host –orig_master_ip=ip –orig_master_port=port
–new_master_host=host –new_master_ip=ip –new_master_port=port\n”;
}
在脚本中绿色背景的代码是新增的，红色背景的代码是注释的。
在主库添加关闭 keepalived 的脚本
vi /root/lvs-admin
case “$1” in
“stop”)
echo;
echo “stop keepalived ……”;
service keepalived stop
;;
“start”)
echo;
echo “start keepalive ……”;
service keepalived start
;;
*)
echo “please input you select!”
;;
esac
chmod 755 /root/lvs-admin
7. 测试主库故障MHA自动把备库提升为主库并把虚拟IP漂移到新的主库（原来的备库）
7.1 在备库停止复制默认有二进制日志没传到备库
mysql> stop slave;
7.2 sysbench 生成测试数据，在主库上的 sbtest 库下创建 sbtest 表，共100万条数据
mysql> create database sbtest;
./sysbench –num-threads=4
–test=oltp –oltp-table-size=1000000 –oltp-read-only=off –init-rng=on –max-requests=0 \
–oltp-dist-type=uniform
–max-time=1800 –mysql-user=root –db-driver=test –mysql-table-engine=innodb
\
–oltp-test-mode=complex prepare
7.3 关闭主库 MHA会自动把备份库提升为新的主库，并把虚拟IP漂移到新的主库（原来的备库）
—关闭主库
mysqladmin -u root shutdown
—在MHA监控节点（备库）实时查看MHA日志
tail -f /masterha/app1/app1.log
Thu Feb 2 23:53:06 2017 – [info] MHA::MasterMonitor
version 0.57.
Thu Feb 2 23:53:07 2017 –
[error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master
192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in
the configuration file!
Thu Feb 2 23:53:07 2017 – [error][/usr/local/share/perl5/MHA/MasterMonitor.pm,
ln427] Error happened on checking configurations. at /usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Thu Feb 2 23:53:07 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring
servers.
Thu Feb 2 23:53:07 2017 – [info] Got exit code 1 (Not
master dead).
Thu Feb 2 23:54:15 2017 – [info] MHA::MasterMonitor
version 0.57.
Thu Feb 2 23:54:16 2017 –
[error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master 192.168.222.156:3306
from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in
the configuration file!
Thu Feb 2 23:54:16 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Thu Feb 2 23:54:16 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Thu Feb 2 23:54:16 2017 – [info] Got exit code 1 (Not
master dead).
Thu Feb 2 23:58:13 2017 – [info] MHA::MasterMonitor
version 0.57.
Thu Feb 2 23:58:14 2017 –
[error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master
192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in
the configuration file!
Thu Feb 2 23:58:14 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Thu Feb 2 23:58:14 2017 – [error][/usr/local/share/perl5/MHA/MasterMonitor.pm,
ln525] Error happened on monitoring servers.
Thu Feb 2 23:58:14 2017 – [info] Got exit code 1 (Not
master dead).
Thu Feb 2 23:59:48 2017 – [info] MHA::MasterMonitor
version 0.57.
Thu Feb 2 23:59:49 2017 – [error][/usr/local/share/perl5/MHA/ServerManager.pm,
ln671] Master 192.168.222.156:3306 from which slave
192.168.222.157(192.168.222.157:330
6) replicates is not defined in
the configuration file!
Thu Feb 2 23:59:49 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Thu Feb 2 23:59:49 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Thu Feb 2 23:59:49 2017 – [info] Got exit code 1 (Not
master dead).
Fri Feb 3 00:04:10 2017 – [info] MHA::MasterMonitor
version 0.57.
Fri Feb 3 00:04:11 2017 –
[error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master
192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in
the configuration file!
Fri Feb 3 00:04:11 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Fri Feb 3 00:04:11 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Fri Feb 3 00:04:11 2017 – [info] Got exit code 1 (Not
master dead).
Fri Feb 3 00:09:04 2017 – [info] MHA::MasterMonitor
version 0.57.
Fri Feb 3 00:09:05 2017 –
[error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master
192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in
the configuration file!
Fri Feb 3 00:09:05 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Fri Feb 3 00:09:05 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Fri Feb 3 00:09:05 2017 – [info] Got exit code 1 (Not
master dead).
Fri Feb 3 00:13:13 2017 – [info] MHA::MasterMonitor
version 0.57.
Fri Feb 3 00:13:14 2017 – [error][/usr/local/share/perl5/MHA/ServerManager.pm,
ln671] Master 192.168.222.156:3306 from which slave
192.168.222.157(192.168.222.157:330
6) replicates is not defined in
the configuration file!
Fri Feb 3 00:13:14 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Fri Feb 3 00:13:14 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Fri Feb 3 00:13:14 2017 – [info] Got exit code 1 (Not
master dead).
Fri Feb 3 00:20:21 2017 – [info] MHA::MasterMonitor
version 0.57.
Fri Feb 3 00:20:23 2017 –
[error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master
192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in
the configuration file!
Fri Feb 3 00:20:23 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Fri Feb 3 00:20:23 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Fri Feb 3 00:20:23 2017 – [info] Got exit code 1 (Not
master dead).
Fri Feb 3 00:21:54 2017 – [info] MHA::MasterMonitor
version 0.57.
Fri Feb 3 00:21:55 2017 – [info] GTID failover mode =
0
Fri Feb 3 00:21:55 2017 – [info] Dead Servers:
Fri Feb 3 00:21:55 2017 – [info] Alive Servers:
Fri Feb 3 00:21:55 2017 – [info] 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:21:55 2017 – [info] 192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:21:55 2017 – [info] Alive Slaves:
Fri Feb 3 00:21:55 2017 – [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version
between slaves) log-bin:enabled
Fri Feb 3 00:21:55 2017 – [info] Replicating from
192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:21:55 2017 – [info] Primary candidate for the new Master
(candidate_master is set)
Fri Feb 3 00:21:55 2017 – [info] Current Alive
Master: 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:21:55 2017 – [info] Checking slave
configurations..
Fri Feb 3 00:21:55 2017 – [info] Checking replication
filtering settings..
Fri Feb 3 00:21:55 2017 – [info] binlog_do_db= , binlog_ignore_db=
Fri Feb 3 00:21:55 2017 – [info] Replication filtering check ok.
Fri Feb 3 00:21:55 2017 – [info] GTID (with auto-pos)
is not supported
Fri Feb 3 00:21:55 2017 – [info] Starting SSH
connection tests..
Fri Feb 3 00:21:58 2017 – [info] All SSH connection
tests passed successfully.
Fri Feb 3 00:21:58 2017 – [info] Checking MHA Node
version..
Fri Feb 3 00:21:58 2017 – [info] Version check ok.
Fri Feb 3 00:21:58 2017 – [info] Checking SSH
publickey authentication settings on the current master..
Fri Feb 3 00:21:59 2017 – [info] HealthCheck: SSH to
192.168.222.156 is reachable.
Fri Feb 3 00:21:59 2017 – [info] Master MHA Node
version is 0.57.
Fri Feb 3 00:21:59 2017 – [info] Checking recovery
script configurations on 192.168.222.156(192.168.222.156:3306)..
Fri Feb 3 00:21:59 2017 – [info] Executing command: save_binary_logs
–command=test –start_pos=4
–binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data –output_file=
/tmp/save_binary_logs_test
–manager_version=0.57 –start_file=mysql-bin.000068
Fri Feb 3 00:21:59 2017 – [info] Connecting to
root@192.168.222.156(192.168.222.156:22)..
Creating /tmp if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /app/oracle/oradata2/mysql-5.5.37/data, up to
mysql-bin.000068
Fri Feb 3 00:21:59 2017 – [info] Binlog setting check
done.
Fri Feb 3 00:21:59 2017 – [info] Checking SSH
publickey authentication and checking recovery script configurations on all
alive slave servers..
Fri Feb 3 00:21:59 2017 – [info] Executing command : apply_diff_relay_logs
–command=test –slave_user=’root’ –slave_host=192.168.222.157
–slave_ip=192.168.222.15
7 –slave_port=3306 –workdir=/tmp
–target_version=5.6.27-log –manager_version=0.57 –relay_log_info=/usr/local/mysql/data/relay-log.info –relay_dir=/usr/local/mys
ql/data/ –slave_pass=xxx
Fri Feb 3 00:21:59 2017 – [info] Connecting to
root@192.168.222.157(192.168.222.157:22)..
Checking slave recovery environment settings..
Opening /usr/local/mysql/data/relay-log.info … ok.
Relay log found at /usr/local/mysql/data, up to relay-bin.000002
Temporary relay log file is /usr/local/mysql/data/relay-bin.000002
Testing mysql connection and privileges..Warning: Using a password on
the command line interface can be insecure.
done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Fri Feb 3 00:22:00 2017 – [info] Slaves settings
check done.
Fri Feb 3 00:22:00 2017 – [info]
192.168.222.156(192.168.222.156:3306)
(current master)
+–192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:22:00 2017 – [info] Checking
master_ip_failover_script status:
Fri Feb 3 00:22:00 2017 – [info] /usr/local/bin/master_ip_failover
–command=status –ssh_user=root –orig_master_host=192.168.222.156
–orig_master_ip=192.168.222.
156 –orig_master_port=3306
Fri Feb 3 00:22:00 2017 – [info] OK.
Fri Feb 3 00:22:00 2017 – [warning] shutdown_script
is not defined.
Fri Feb 3 00:22:00 2017 – [info] Set master ping
interval 1 seconds.
Fri Feb 3 00:22:00 2017 – [info] Set secondary check
script: /usr/local/bin/masterha_secondary_check -s mysqldb2 -s oradb
–user=root –master_host=oradb –master_ip=
‘192.168.222.156’
–master_port=3306
Fri Feb 3 00:22:00 2017 – [info] Starting ping health
check on 192.168.222.156(192.168.222.156:3306)..
Fri Feb 3 00:22:00 2017 – [info] Ping(SELECT)
succeeded, waiting until MySQL doesn’t respond..
Fri Feb 3 00:25:06 2017 – [warning] Got error on
MySQL select ping: 2013 (Lost connection to MySQL server during query)
Fri Feb 3 00:25:06 2017 – [info] Executing secondary
network check script: /usr/local/bin/masterha_secondary_check -s mysqldb2 -s
oradb –user=root –master_host=orad
b –master_ip=’192.168.222.156′
–master_port=3306 –user=root –master_host=192.168.222.156 –master_ip=192.168.222.156 –master_port=3306 –master_user=root –ma
ster_password=123456
–ping_type=SELECT
Fri Feb 3 00:25:06 2017 – [info] Executing SSH check
script: save_binary_logs –command=test –start_pos=4 –binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data
–outpu
t_file=/tmp/save_binary_logs_test
–manager_version=0.57 –binlog_prefix=mysql-bin
Fri Feb 3 00:25:07 2017 – [warning] Got error on
MySQL connect: 2003 (Can’t connect to MySQL server on ‘192.168.222.156’ (111))
Fri Feb 3 00:25:07 2017 – [warning] Connection failed
2 time(s)..
Monitoring server mysqldb2 is
reachable, Master is not reachable from mysqldb2. OK.
Fri Feb 3 00:25:08 2017 – [warning] Got error on
MySQL connect: 2003 (Can’t connect to MySQL server on ‘192.168.222.156’ (111))
Fri Feb 3 00:25:08 2017 – [warning] Connection failed
3 time(s)..
Monitoring server oradb is
reachable, Master is not reachable from oradb. OK.
Fri Feb 3 00:25:08 2017 – [info] Master is not
reachable from all other monitoring servers. Failover should start.
Fri Feb 3 00:25:08 2017 – [info] HealthCheck: SSH to
192.168.222.156 is reachable.
Fri Feb 3 00:25:09 2017 – [warning] Got error on
MySQL connect: 2003 (Can’t connect to MySQL server on ‘192.168.222.156’ (111))
Fri Feb 3 00:25:09 2017 – [warning] Connection failed
4 time(s)..
Fri Feb 3 00:25:09 2017 – [warning] Master is not
reachable from health checker!
Fri Feb 3 00:25:09 2017 – [warning] Master
192.168.222.156(192.168.222.156:3306) is not reachable!
Fri Feb 3 00:25:09 2017 – [warning] SSH is reachable.
Fri Feb 3 00:25:09 2017 – [info] Connecting to a
master server failed. Reading configuration file /etc/masterha_default.cnf and
/etc/masterha/app1.cnf again, and tryi
ng to connect to all servers to
check server status..
Fri Feb 3 00:25:09 2017 – [warning] Global
configuration file /etc/masterha_default.cnf not found. Skipping.
Fri Feb 3 00:25:09 2017 – [info] Reading application
default configuration from /etc/masterha/app1.cnf..
Fri Feb 3 00:25:09 2017 – [info] Reading server
configuration from /etc/masterha/app1.cnf..
Fri Feb 3 00:25:10 2017 – [warning] SQL Thread is
stopped(no error) on 192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:25:10 2017 – [info] GTID failover mode =
0
Fri Feb 3 00:25:10 2017 – [info] Dead Servers:
Fri Feb 3 00:25:10 2017 – [info] 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:10 2017 – [info] Alive Servers:
Fri Feb 3 00:25:10 2017 – [info] 192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:25:10 2017 – [info] Alive Slaves:
Fri Feb 3 00:25:10 2017 – [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version
between slaves) log-bin:enabled
Fri Feb 3 00:25:10 2017 – [info] Replicating from
192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:10 2017 – [info] Primary candidate for the new Master
(candidate_master is set)
Fri Feb 3 00:25:10 2017 – [info] Checking slave
configurations..
Fri Feb 3 00:25:10 2017 – [info] Checking replication
filtering settings..
Fri Feb 3 00:25:10 2017 – [info] Replication filtering check ok.
Fri Feb 3 00:25:10 2017 – [info] Master is down!
Fri Feb 3 00:25:10 2017 – [info] Terminating
monitoring script.
Fri Feb 3 00:25:10 2017 – [info] Got exit code 20
(Master dead).
Fri Feb 3 00:25:10 2017 – [info] MHA::MasterFailover
version 0.57.
Fri Feb 3 00:25:10 2017 – [info] Starting master
failover.
Fri Feb 3 00:25:10 2017 – [info]
Fri Feb 3 00:25:10 2017 – [info] * Phase 1:
Configuration Check Phase..
Fri Feb 3 00:25:10 2017 – [info]
Fri Feb 3 00:25:11 2017 – [warning] SQL Thread is
stopped(no error) on 192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:25:11 2017 – [info] GTID failover mode =
0
Fri Feb 3 00:25:11 2017 – [info] Dead Servers:
Fri Feb 3 00:25:11 2017 – [info] 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:11 2017 – [info] Checking master
reachability via MySQL(double check)…
Fri Feb 3 00:25:11 2017 – [info] ok.
Fri Feb 3 00:25:11 2017 – [info] Alive Servers:
Fri Feb 3 00:25:11 2017 – [info] 192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:25:11 2017 – [info] Alive Slaves:
Fri Feb 3 00:25:11 2017 – [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version
between slaves) log-bin:enabled
Fri Feb 3 00:25:11 2017 – [info] Replicating from
192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:11 2017 – [info] Primary candidate for the new Master
(candidate_master is set)
Fri Feb 3 00:25:11 2017 – [info] Starting SQL thread on
192.168.222.157(192.168.222.157:3306) ..
Fri Feb 3 00:25:11 2017 – [info] done.
Fri Feb 3 00:25:11 2017 – [info] Starting Non-GTID
based failover.
Fri Feb 3 00:25:11 2017 – [info]
Fri Feb 3 00:25:11 2017 – [info] ** Phase 1:
Configuration Check Phase completed.
Fri Feb 3 00:25:11 2017 – [info]
Fri Feb 3 00:25:11 2017 – [info] * Phase 2: Dead
Master Shutdown Phase..
Fri Feb 3 00:25:11 2017 – [info]
Fri Feb 3 00:25:11 2017 – [info] Forcing shutdown so
that applications never connect to the current master..
Fri Feb 3 00:25:11 2017 – [info] Executing master IP
deactivation script:
Fri Feb 3 00:25:11 2017 – [info] /usr/local/bin/master_ip_failover
–orig_master_host=192.168.222.156 –orig_master_ip=192.168.222.156
–orig_master_port=3306 –com
mand=stopssh –ssh_user=root
Fri Feb 3 00:25:11 2017 – [info] done.
Fri Feb 3 00:25:11 2017 – [warning] shutdown_script
is not set. Skipping explicit shutting down of the dead master.
Fri Feb 3 00:25:11 2017 – [info] * Phase 2: Dead
Master Shutdown Phase completed.
Fri Feb 3 00:25:11 2017 – [info]
Fri Feb 3 00:25:11 2017 – [info] * Phase 3: Master
Recovery Phase..
Fri Feb 3 00:25:11 2017 – [info]
Fri Feb 3 00:25:11 2017 – [info] * Phase 3.1: Getting
Latest Slaves Phase..
Fri Feb 3 00:25:11 2017 – [info]
Fri Feb 3 00:25:11 2017 – [info] The latest binary
log file/position on all slaves is mysql-bin.000068:530
Fri Feb 3 00:25:11 2017 – [info] Latest slaves
(Slaves that received relay log files to the latest):
Fri Feb 3 00:25:11 2017 – [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version
between slaves) log-bin:enabled
Fri Feb 3 00:25:11 2017 – [info] Replicating from
192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:11 2017 – [info] Primary candidate for the new Master
(candidate_master is set)
Fri Feb 3 00:25:11 2017 – [info] The oldest binary
log file/position on all slaves is mysql-bin.000068:530
Fri Feb 3 00:25:11 2017 – [info] Oldest slaves:
Fri Feb 3 00:25:11 2017 – [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version
between slaves) log-bin:enabled
Fri Feb 3 00:25:11 2017 – [info] Replicating from
192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:11 2017 – [info] Primary candidate for the new Master
(candidate_master is set)
Fri Feb 3 00:25:11 2017 – [info]
Fri Feb 3 00:25:11 2017 – [info] * Phase 3.2: Saving
Dead Master’s Binlog Phase..
Fri Feb 3 00:25:11 2017 – [info]
Fri Feb 3 00:25:12 2017 – [info] Fetching dead
master’s binary logs..
Fri Feb 3 00:25:12 2017 – [info] Executing command on
the dead master 192.168.222.156(192.168.222.156:3306): save_binary_logs
–command=save –start_file=mysql-bin.00
0068 –start_pos=530
–binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data
–output_file=/tmp/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog
–hand
le_raw_binlog=1
–disable_log_bin=0 –manager_version=0.57
Creating /tmp if not exists.. ok.
Concat binary/relay logs from mysql-bin.000068
pos 530 to mysql-bin.000068 EOF into
/tmp/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog ..
Binlog Checksum enabled
Dumping binlog format description event, from position 0 to 120.. ok.
Dumping effective binlog data from
/app/oracle/oradata2/mysql-5.5.37/data/mysql-bin.000068 position 530 to
tail(61261921).. ok.
Binlog Checksum enabled
Concat succeeded.
Fri Feb 3 00:25:16 2017 – [info] scp from
root@192.168.222.156:/tmp/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog
to local:/masterha/app1/saved_
master_binlog_from_192.168.222.156_3306_20170203002510.binlog
succeeded.
Fri Feb 3 00:25:17 2017 – [info] HealthCheck: SSH to
192.168.222.157 is reachable.
Fri Feb 3 00:25:17 2017 – [info]
Fri Feb 3 00:25:17 2017 – [info] * Phase 3.3:
Determining New Master Phase..
Fri Feb 3 00:25:17 2017 – [info]
Fri Feb 3 00:25:17 2017 – [info] Finding the latest
slave that has all relay logs for recovering other slaves..
Fri Feb 3 00:25:17 2017 – [info] All slaves received
relay logs to the same position. No need to resync each other.
Fri Feb 3 00:25:17 2017 – [info] Searching new master
from slaves..
Fri Feb 3 00:25:17 2017 – [info] Candidate masters from the configuration file:
Fri Feb 3 00:25:17 2017 – [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version
between slaves) log-bin:enabled
Fri Feb 3 00:25:17 2017 – [info] Replicating from
192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:17 2017 – [info] Primary candidate for the new Master
(candidate_master is set)
Fri Feb 3 00:25:17 2017 – [info] Non-candidate masters:
Fri Feb 3 00:25:17 2017 – [info] Searching from candidate_master slaves which
have received the latest relay log events..
Fri Feb 3 00:25:17 2017 – [info] New master is
192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:25:17 2017 – [info] Starting master
failover..
Fri Feb 3 00:25:17 2017 – [info]
From:
192.168.222.156(192.168.222.156:3306)
(current master)
+–192.168.222.157(192.168.222.157:3306)
To:
192.168.222.157(192.168.222.157:3306)
(new master)
Fri Feb 3 00:25:17 2017 – [info]
Fri Feb 3 00:25:17 2017 – [info] * Phase 3.3: New
Master Diff Log Generation Phase..
Fri Feb 3 00:25:17 2017 – [info]
Fri Feb 3 00:25:17 2017 – [info] This server has all relay logs. No need to
generate diff files from the latest slave.
Fri Feb 3 00:25:17 2017 – [info] Sending binlog..
Fri Feb 3 00:25:20 2017 – [info] scp from local:/masterha/app1/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog
to root@192.168.222.157:/tmp/saved_
master_binlog_from_192.168.222.156_3306_20170203002510.binlog
succeeded.
Fri Feb 3 00:25:20 2017 – [info]
Fri Feb 3 00:25:20 2017 – [info] * Phase 3.4: Master
Log Apply Phase..
Fri Feb 3 00:25:20 2017 – [info]
Fri Feb 3 00:25:20 2017 – [info] *NOTICE: If any
error happens from this phase, manual recovery is needed.
Fri Feb 3 00:25:20 2017 – [info] Starting recovery on
192.168.222.157(192.168.222.157:3306)..
Fri Feb 3 00:25:20 2017 – [info] Generating diffs succeeded.
Fri Feb 3 00:25:20 2017 – [info] Waiting until all
relay logs are applied.
Fri Feb 3 00:25:20 2017 – [info] done.
Fri Feb 3 00:25:20 2017 – [info] Getting slave status..
Fri Feb 3 00:25:20 2017 – [info] This
slave(192.168.222.157)’s Exec_Master_Log_Pos equals to
Read_Master_Log_Pos(mysql-bin.000068:530). No need to recover from Exec_M
aster_Log_Pos.
Fri Feb 3 00:25:20 2017 – [info] Connecting to the
target slave host 192.168.222.157, running recover script..
Fri Feb 3 00:25:20 2017 – [info] Executing command:
apply_diff_relay_logs –command=apply –slave_user=’root’
–slave_host=192.168.222.157 –slave_ip=192.168.222.157
–slave_port=3306 –apply_files=/tmp/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog
–workdir=/tmp –target_version=5.6.27-log –timestamp=201702
03002510 –handle_raw_binlog=1
–disable_log_bin=0 –manager_version=0.57 –slave_pass=xxx
Fri Feb 3 00:26:48 2017 – [info]
MySQL client version is 5.6.27.
Using –binary-mode.
Applying differential binary/relay
log files
/tmp/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog on
192.168.222.157:3306. This may take long time.
..
Applying log files succeeded.
Fri Feb 3 00:26:48 2017 – [info] All relay logs were successfully applied.
Fri Feb 3 00:26:48 2017 – [info] Getting new master’s
binlog name and position..
Fri Feb 3 00:26:48 2017 – [info] mysql-bin.000002:196188901
Fri Feb 3 00:26:48 2017 – [info] All other slaves should start replication
from here. Statement should be: CHANGE MASTER TO MASTER_HOST=’192.168.222.157′,
MASTER_POR
T=3306,
MASTER_LOG_FILE=’mysql-bin.000002′, MASTER_LOG_POS=196188901,
MASTER_USER=’repl’, MASTER_PASSWORD=’xxx’;
Fri Feb 3 00:26:48 2017 – [info] Executing master IP
activate script:
Fri Feb 3 00:26:48 2017 – [info] /usr/local/bin/master_ip_failover
–command=start –ssh_user=root –orig_master_host=192.168.222.156
–orig_master_ip=192.168.222.1
56 –orig_master_port=3306 –new_master_host=192.168.222.157
–new_master_ip=192.168.222.157 –new_master_port=3306
–new_master_user=’root’ –new_master_password=xx
x
Set read_only=0 on the new master.
Creating app user on the new
master..
sh:
sshroot@192.168.222.156./lvs-admin stop: No such file or directory
Fri Feb 3 00:26:49 2017 – [info] OK.
Fri Feb 3 00:26:49 2017 – [info] ** Finished master
recovery successfully.
Fri Feb 3 00:26:49 2017 – [info] * Phase 3: Master
Recovery Phase completed.
Fri Feb 3 00:26:49 2017 – [info]
Fri Feb 3 00:26:49 2017 – [info] * Phase 4: Slaves
Recovery Phase..
Fri Feb 3 00:26:49 2017 – [info]
Fri Feb 3 00:26:49 2017 – [info] * Phase 4.1:
Starting Parallel Slave Diff Log Generation Phase..
Fri Feb 3 00:26:49 2017 – [info]
Fri Feb 3 00:26:49 2017 – [info] Generating relay
diff files from the latest slave succeeded.
Fri Feb 3 00:26:49 2017 – [info]
Fri Feb 3 00:26:49 2017 – [info] * Phase 4.2:
Starting Parallel Slave Log Apply Phase..
Fri Feb 3 00:26:49 2017 – [info]
Fri Feb 3 00:26:49 2017 – [info] All new slave
servers recovered successfully.
Fri Feb 3 00:26:49 2017 – [info]
Fri Feb 3 00:26:49 2017 – [info] * Phase 5: New
master cleanup phase..
Fri Feb 3 00:26:49 2017 – [info]
Fri Feb 3 00:26:49 2017 – [info] Resetting slave info
on the new master..
Fri Feb 3 00:26:49 2017 – [info] 192.168.222.157: Resetting slave info
succeeded.
Fri Feb 3 00:26:49 2017 – [info] Master failover to
192.168.222.157(192.168.222.157:3306) completed successfully.
Fri Feb 3 00:26:49 2017 – [info] Deleted server1
entry from /etc/masterha/app1.cnf .
Fri Feb 3 00:26:49 2017 – [info]
—– Failover Report —–
app1: MySQL Master failover
192.168.222.156(192.168.222.156:3306) to 192.168.222.157(192.168.222.157:3306)
succeeded
Master 192.168.222.156(192.168.222.156:3306)
is down!
Check MHA Manager logs at
mysqldb2:/masterha/app1/app1.log for details.
Started automated(non-interactive)
failover.
Invalidated master IP address on
192.168.222.156(192.168.222.156:3306)
The latest slave
192.168.222.157(192.168.222.157:3306) has all relay logs for recovery.
Selected
192.168.222.157(192.168.222.157:3306) as a new master.
192.168.222.157(192.168.222.157:3306):
OK: Applying all logs succeeded.
192.168.222.157(192.168.222.157:3306):
OK: Activated master IP address.
Generating relay diff files from
the latest slave succeeded.
192.168.222.157(192.168.222.157:3306):
Resetting slave info succeeded.
Master failover to
192.168.222.157(192.168.222.157:3306) completed successfully.
三、部署 MHA遇到的问题
问题：
[root@localhost
mha]# ./install.sh
–> Working on install
Fetching
http://www.cpan.org/authors/id/D/DA/DAGOLDEN/install-0.01.tar.gz … OK
Configuring install-0.01 … OK
Building and testing install-0.01
… FAIL
! Installing install failed. See
/root/.cpanm/work/1485934105.3288/build.log for details. Retry with –force to
force install it.
–> Working on DBD::mysql
Fetching
http://www.cpan.org/authors/id/M/MI/MICHIELB/DBD-mysql-4.041.tar.gz … OK
==> Found dependencies: DBI
–> Working on DBI
Fetching http://www.cpan.org/authors/id/T/TI/TIMB/DBI-1.636.tar.gz
… OK
Configuring DBI-1.636 … N/A
! Configure failed for DBI-1.636.
See /root/.cpanm/work/1485934125.3353/build.log for details.
! Installing the dependencies
failed: Module ‘DBI’ is not installed
! Bailing out the installation for
DBD-mysql-4.041.
问题：
/usr/bin/ld: cannot find -lz
usr/bin/ld: cannot find -lz
collect2: ld returned 1 exit
status
make: ***
[blib/arch/auto/DBD/mysql/mysql.so] Error 1
-> FAIL Installing DBD::mysql
failed. See /root/.cpanm/work/1485938925.8246/build.log for details. Retry with
–force to force install it.
2 distributions installed
处理：
yum -y install *zlib-dev*
[root@localhost mha]# ./install.sh
install is up to date. (0.01)
–> Working on DBD::mysql
Fetching http://www.cpan.org/authors/id/M/MI/MICHIELB/DBD-mysql-4.041.tar.gz
… OK
Configuring DBD-mysql-4.041 … OK
Building and testing
DBD-mysql-4.041 … OK
Successfully installed
DBD-mysql-4.041 (upgraded from 4.013)
1 distribution installed
[root@localhost
mha4mysql-node-0.53]# perl Makefile.PL
*** Module::AutoInstall version
1.03
*** Checking for Perl
dependencies…
[Core Features]
– DBI …loaded. (1.609)
– DBD::mysql …loaded. (4.041)
*** Module::AutoInstall
configuration finished.
Checking if your kit is
complete…
Looks good
Generating a Unix-style Makefile
Writing Makefile for
mha4mysql::node
Writing MYMETA.yml and MYMETA.json
[root@localhost
mha4mysql-node-0.53]# make
cp lib/MHA/BinlogManager.pm
blib/lib/MHA/BinlogManager.pm
cp lib/MHA/BinlogPosFindManager.pm
blib/lib/MHA/BinlogPosFindManager.pm
cp lib/MHA/BinlogPosFinderXid.pm
blib/lib/MHA/BinlogPosFinderXid.pm
cp lib/MHA/BinlogHeaderParser.pm
blib/lib/MHA/BinlogHeaderParser.pm
cp lib/MHA/BinlogPosFinder.pm
blib/lib/MHA/BinlogPosFinder.pm
cp lib/MHA/BinlogPosFinderElp.pm
blib/lib/MHA/BinlogPosFinderElp.pm
cp lib/MHA/NodeUtil.pm
blib/lib/MHA/NodeUtil.pm
cp lib/MHA/SlaveUtil.pm
blib/lib/MHA/SlaveUtil.pm
cp lib/MHA/NodeConst.pm
blib/lib/MHA/NodeConst.pm
cp bin/apply_diff_relay_logs blib/script/apply_diff_relay_logs
“/usr/bin/perl”
“-Iinc” -MExtUtils::MY -e ‘MY->fixin(shift)’ —
blib/script/apply_diff_relay_logs
cp bin/filter_mysqlbinlog
blib/script/filter_mysqlbinlog
“/usr/bin/perl”
“-Iinc” -MExtUtils::MY -e ‘MY->fixin(shift)’ — blib/script/filter_mysqlbinlog
cp bin/purge_relay_logs
blib/script/purge_relay_logs
“/usr/bin/perl”
“-Iinc” -MExtUtils::MY -e ‘MY->fixin(shift)’ —
blib/script/purge_relay_logs
cp bin/save_binary_logs
blib/script/save_binary_logs
“/usr/bin/perl”
“-Iinc” -MExtUtils::MY -e ‘MY->fixin(shift)’ —
blib/script/save_binary_logs
Manifying 4 pod documents
[root@localhost
mha4mysql-node-0.53]# make install
Manifying 4 pod documents
Installing
/usr/local/share/perl5/MHA/NodeUtil.pm
Installing
/usr/local/share/perl5/MHA/BinlogPosFindManager.pm
Installing
/usr/local/share/perl5/MHA/NodeConst.pm
Installing
/usr/local/share/perl5/MHA/BinlogPosFinder.pm
Installing
/usr/local/share/perl5/MHA/BinlogHeaderParser.pm
Installing
/usr/local/share/perl5/MHA/BinlogPosFinderXid.pm
Installing
/usr/local/share/perl5/MHA/BinlogPosFinderElp.pm
Installing
/usr/local/share/perl5/MHA/BinlogManager.pm
Installing
/usr/local/share/perl5/MHA/SlaveUtil.pm
Installing
/usr/local/share/man/man1/purge_relay_logs.1
Installing /usr/local/share/man/man1/filter_mysqlbinlog.1
Installing
/usr/local/share/man/man1/save_binary_logs.1
Installing
/usr/local/share/man/man1/apply_diff_relay_logs.1
Installing
/usr/local/bin/apply_diff_relay_logs
Installing
/usr/local/bin/save_binary_logs
Installing /usr/local/bin/filter_mysqlbinlog
Installing
/usr/local/bin/purge_relay_logs
Appending installation info to
/usr/lib64/perl5/perllocal.pod
问题10：从 https://github.com/yoshinorim/mha4mysql-node 下载 mha4mysql-node 安装
[root@mysqldb2
mha4mysql-node-master]# perl Makefile.PL
Can’t locate inc/Module/Install.pm
in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5
/usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5
/usr/share/perl5 .) at Makefile.PL line 1.
BEGIN failed–compilation aborted
at Makefile.PL line 1.
处理：
—查看 Makefile.PL 第一行的内容
[root@mysqldb2 mha4mysql-node-master]# vi Makefile.PL
use inc::Module::Install;
—安装模块
cpanm inc::Module::Install
[root@mysqldb2 mha4mysql-node-master]# perl Makefile.PL
include /root/olm/mha/mha4mysql-node-master/inc/Module/Install.pm
include inc/Module/Install/Metadata.pm
include inc/Module/Install/Base.pm
include inc/Module/Install/Makefile.pm
include inc/Module/Install/Scripts.pm
include inc/Module/Install/AutoInstall.pm
include inc/Module/Install/Include.pm
include inc/Module/AutoInstall.pm
*** Module::AutoInstall version 1.17
*** Checking for Perl dependencies…
[Core Features]
– DBI …loaded.
(1.609)
– DBD::mysql …loaded. (4.041)
*** Module::AutoInstall configuration finished.
include inc/Module/Install/WriteAll.pm
include inc/Module/Install/Win32.pm
include inc/Module/Install/Can.pm
include inc/Module/Install/Fetch.pm
Checking if your kit is complete…
Warning: the following files are missing in your kit:
META.yml
Please inform the author.
Generating a Unix-style Makefile
Writing Makefile for mha4mysql::node
Writing MYMETA.yml and MYMETA.json
Writing META.yml
[root@mysqldb2 mha4mysql-node-master]# make
cp lib/MHA/BinlogManager.pm blib/lib/MHA/BinlogManager.pm
cp lib/MHA/BinlogPosFindManager.pm
blib/lib/MHA/BinlogPosFindManager.pm
cp lib/MHA/BinlogPosFinderXid.pm blib/lib/MHA/BinlogPosFinderXid.pm
cp lib/MHA/BinlogHeaderParser.pm blib/lib/MHA/BinlogHeaderParser.pm
cp lib/MHA/BinlogPosFinder.pm blib/lib/MHA/BinlogPosFinder.pm
cp lib/MHA/BinlogPosFinderElp.pm blib/lib/MHA/BinlogPosFinderElp.pm
cp lib/MHA/NodeUtil.pm blib/lib/MHA/NodeUtil.pm
cp lib/MHA/SlaveUtil.pm blib/lib/MHA/SlaveUtil.pm
cp lib/MHA/NodeConst.pm blib/lib/MHA/NodeConst.pm
cp bin/apply_diff_relay_logs blib/script/apply_diff_relay_logs
“/usr/bin/perl” “-Iinc” -MExtUtils::MY -e
‘MY->fixin(shift)’ — blib/script/apply_diff_relay_logs
cp bin/filter_mysqlbinlog blib/script/filter_mysqlbinlog
“/usr/bin/perl” “-Iinc” -MExtUtils::MY -e
‘MY->fixin(shift)’ — blib/script/filter_mysqlbinlog
cp bin/purge_relay_logs blib/script/purge_relay_logs
“/usr/bin/perl” “-Iinc” -MExtUtils::MY -e
‘MY->fixin(shift)’ — blib/script/purge_relay_logs
cp bin/save_binary_logs blib/script/save_binary_logs
“/usr/bin/perl” “-Iinc” -MExtUtils::MY -e
‘MY->fixin(shift)’ — blib/script/save_binary_logs
Manifying 4 pod documents
[root@mysqldb2 mha4mysql-node-master]# make install
Manifying 4 pod documents
Installing /usr/local/share/perl5/MHA/NodeUtil.pm
Installing /usr/local/share/perl5/MHA/NodeConst.pm
Installing /usr/local/share/perl5/MHA/BinlogHeaderParser.pm
Installing /usr/local/share/perl5/MHA/BinlogManager.pm
Installing /usr/local/share/perl5/MHA/SlaveUtil.pm
Installing /usr/local/share/man/man1/purge_relay_logs.1
Installing /usr/local/share/man/man1/filter_mysqlbinlog.1
Installing /usr/local/share/man/man1/save_binary_logs.1
Installing /usr/local/share/man/man1/apply_diff_relay_logs.1
Installing /usr/local/bin/apply_diff_relay_logs
Installing /usr/local/bin/save_binary_logs
Installing /usr/local/bin/purge_relay_logs
Appending installation info to /usr/lib64/perl5/perllocal.pod
问题11：处理完问题10后
[root@mysqldb2 debian]# /usr/local/bin/masterha_check_ssh
–conf=/etc/masterha/app1.cnf
“NI_NUMERICHOST” is not exported by the Socket module
“getaddrinfo” is
not exported by the Socket module
“getnameinfo” is
not exported by the Socket module
Can’t continue after import errors at
/usr/local/share/perl5/MHA/NodeUtil.pm line 29.
BEGIN failed–compilation aborted at /usr/local/share/perl5/MHA/NodeUtil.pm
line 29.
Compilation failed in require at
/usr/local/share/perl5/MHA/SlaveUtil.pm line 27.
BEGIN failed–compilation aborted at
/usr/local/share/perl5/MHA/SlaveUtil.pm line 27.
Compilation failed in require at /usr/local/share/perl5/MHA/DBHelper.pm
line 26.
BEGIN failed–compilation aborted at
/usr/local/share/perl5/MHA/DBHelper.pm line 26.
Compilation failed in require at
/usr/local/share/perl5/MHA/HealthCheck.pm line 30.
BEGIN failed–compilation aborted at /usr/local/share/perl5/MHA/HealthCheck.pm
line 30.
Compilation failed in require at
/usr/local/share/perl5/MHA/Server.pm line 28.
BEGIN failed–compilation aborted at
/usr/local/share/perl5/MHA/Server.pm line 28.
Compilation failed in require at /usr/local/share/perl5/MHA/Config.pm
line 29.
BEGIN failed–compilation aborted at
/usr/local/share/perl5/MHA/Config.pm line 29.
Compilation failed in require at
/usr/local/share/perl5/MHA/SSHCheck.pm line 32.
BEGIN failed–compilation aborted at /usr/local/share/perl5/MHA/SSHCheck.pm
line 32.
Compilation failed in require at /usr/local/bin/masterha_check_ssh
line 25.
BEGIN failed–compilation aborted at
/usr/local/bin/masterha_check_ssh line 25.
处理：
[root@mysqldb2 mha4mysql-node-master]# cpanm Socket
–> Working on Socket
Fetching
http://www.cpan.org/authors/id/P/PE/PEVANS/Socket-2.024.tar.gz … OK
==> Found dependencies: ExtUtils::Constant
–> Working on ExtUtils::Constant
Fetching
http://www.cpan.org/authors/id/N/NW/NWCLARK/ExtUtils-Constant-0.23.tar.gz …
OK
Configuring ExtUtils-Constant-0.16 … OK
Building and testing ExtUtils-Constant-0.23 … OK
Successfully installed ExtUtils-Constant-0.23 (upgraded from 0.22)
Configuring Socket-2.024 … OK
Building and testing Socket-2.024 … OK
Successfully installed Socket-2.024 (upgraded from 1.82)
2 distributions installed
[root@mysqldb2 mha4mysql-manager-master]#
/usr/local/bin/masterha_check_ssh –conf=/etc/masterha/app1.cnf
Wed Feb 1 19:41:34 2017 –
[warning] Global configuration file /etc/masterha_default.cnf not found.
Skipping.
Wed Feb 1 19:41:34 2017 –
[info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 19:41:34 2017 –
[info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 19:41:34 2017 –
[info] Starting SSH connection tests..
Wed Feb 1 19:41:51 2017 –
[debug]
Wed Feb 1 19:41:35 2017 –
[debug] Connecting via SSH from
root@192.168.222.157(192.168.222.157:22) to
root@192.168.222.156(192.168.222.156:22)..
Wed Feb 1 19:41:50 2017 –
[debug] ok.
Wed Feb 1 19:41:55 2017 –
[debug]
Wed Feb 1 19:41:34 2017 –
[debug] Connecting via SSH from
root@192.168.222.156(192.168.222.156:22) to
root@192.168.222.157(192.168.222.157:22)..
Wed Feb 1 19:41:55 2017 –
[debug] ok.
Wed Feb 1 19:41:55 2017 –
[info] All SSH connection tests passed successfully.
问题12：
[root@mysqldb2 mha]# masterha_check_repl
–conf=/etc/masterha/app1.cnf
Wed Feb 1 20:04:52 2017 –
[warning] Global configuration file /etc/masterha_default.cnf not found.
Skipping.
Wed Feb 1 20:04:52 2017 –
[info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:04:52 2017 –
[info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:04:52 2017 –
[info] MHA::MasterMonitor version 0.57.
Creating directory /masterha/app1.. done.
Wed Feb 1 20:04:57 2017 –
[info] GTID failover mode = 0
Wed Feb 1 20:04:57 2017 –
[info] Dead Servers:
Wed Feb 1 20:04:57 2017 –
[info] Alive Servers:
Wed Feb 1 20:04:57 2017 –
[info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:04:57 2017 –
[info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:04:57 2017 –
[info] Alive Slaves:
Wed Feb 1 20:04:57 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27 (oldest major version between slaves) log-bin:disabled
Wed Feb 1 20:04:57 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:04:57 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Wed Feb 1 20:04:57 2017 –
[info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:04:57 2017 –
[info] Checking slave configurations..
Wed Feb 1 20:04:57 2017 –
[warning] log-bin is not set on slave
192.168.222.157(192.168.222.157:3306). This host cannot be a master.
Wed Feb 1 20:04:57 2017 –
[info] Checking replication filtering settings..
Wed Feb 1 20:04:57 2017 –
[info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:04:57 2017 –
[info] Replication filtering check ok.
Wed Feb 1 20:04:57 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln364] None of slaves can
be master. Check failover configuration file or log-bin settings in my.cnf
Wed Feb 1 20:04:57 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/bin/masterha_check_repl line 48.
Wed Feb 1 20:04:57 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Wed Feb 1 20:04:57 2017 –
[info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
处理：
vi /usr/local/mysql/my.cnf
log_bin=/usr/local/mysql/data/mysql-bin
#relay_log=/usr/local/mysql/data/relay-bin
skip_slave_start
/usr/local/mysql/bin/mysqld_safe –defaults-file=/usr/local/mysql/my.cnf
–user=mysql &
mysql> start slave;
[root@mysqldb2 mysql]# masterha_check_repl
–conf=/etc/masterha/app1.cnf
Wed Feb 1 20:16:43 2017 –
[warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Feb 1 20:16:43 2017 –
[info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:16:43 2017 –
[info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:16:43 2017 –
[info] MHA::MasterMonitor version 0.57.
Wed Feb 1 20:16:44 2017 –
[info] GTID failover mode = 0
Wed Feb 1 20:16:44 2017 –
[info] Dead Servers:
Wed Feb 1 20:16:44 2017 –
[info] Alive Servers:
Wed Feb 1 20:16:44 2017 –
[info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 –
[info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:16:44 2017 –
[info] Alive Slaves:
Wed Feb 1 20:16:44 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Wed Feb 1 20:16:44 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Wed Feb 1 20:16:44 2017 –
[info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 –
[info] Checking slave configurations..
Wed Feb 1 20:16:44 2017 –
[info] read_only=1 is not set on slave
192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:16:44 2017 –
[warning] relay_log_purge=0 is not set
on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:16:44 2017 –
[info] Checking replication filtering settings..
Wed Feb 1 20:16:44 2017 –
[info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:16:44 2017 –
[info] Replication filtering check ok.
Wed Feb 1 20:16:44 2017 –
[info] GTID (with auto-pos) is not supported
Wed Feb 1 20:16:44 2017 –
[info] Starting SSH connection tests..
Wed Feb 1 20:16:50 2017 –
[info] All SSH connection tests passed successfully.
Wed Feb 1 20:16:50 2017 –
[info] Checking MHA Node version..
Wed Feb 1 20:16:51 2017 –
[info] Version check ok.
Wed Feb 1 20:16:51 2017 –
[info] Checking SSH publickey authentication settings on the current master..
Wed Feb 1 20:16:51 2017 –
[info] HealthCheck: SSH to 192.168.222.156 is reachable.
Wed Feb 1 20:16:51 2017 –
[info] Master MHA Node version is 0.57.
Wed Feb 1 20:16:51 2017 –
[info] Checking recovery script configurations on
192.168.222.156(192.168.222.156:3306)..
Wed Feb 1 20:16:51 2017 –
[info] Executing command:
save_binary_logs –command=test –start_pos=4 –binlog_dir=/home/binlog
–output_file=/tmp/save_binary_logs_test –manager_version=0.57
–start_file=mysql-bin.000058
Wed Feb 1 20:16:51 2017 –
[info] Connecting to
root@192.168.222.156(192.168.222.156:22)..
Failed to save binary log: Binlog not found from /home/binlog! If
you got this error at MHA Manager, please set
“master_binlog_dir=/path/to/binlog_directory_of_the_master” correctly
in the MHA Manager’s configuration file and try again.
at
/usr/local/bin/save_binary_logs line 123.
eval {…} called at
/usr/local/bin/save_binary_logs line 70
main::main() called at
/usr/local/bin/save_binary_logs line 66
Wed Feb 1 20:16:51 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln161] Binlog setting
check failed!
Wed Feb 1 20:16:51 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln408] Master
configuration failed.
Wed Feb 1 20:16:51 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/bin/masterha_check_repl line 48.
Wed Feb 1 20:16:51 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Wed Feb 1 20:16:51 2017 –
[info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
问题12已经处理好，但出现了新的问题，命名为问题13
问题13：
[root@mysqldb2 mysql]# masterha_check_repl
–conf=/etc/masterha/app1.cnf
Wed Feb 1 20:16:43 2017 –
[warning] Global configuration file /etc/masterha_default.cnf not found.
Skipping.
Wed Feb 1 20:16:43 2017 –
[info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:16:43 2017 –
[info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:16:43 2017 –
[info] MHA::MasterMonitor version 0.57.
Wed Feb 1 20:16:44 2017 –
[info] GTID failover mode = 0
Wed Feb 1 20:16:44 2017 –
[info] Dead Servers:
Wed Feb 1 20:16:44 2017 –
[info] Alive Servers:
Wed Feb 1 20:16:44 2017 –
[info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 –
[info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:16:44 2017 –
[info] Alive Slaves:
Wed Feb 1 20:16:44 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Wed Feb 1 20:16:44 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Wed Feb 1 20:16:44 2017 –
[info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 –
[info] Checking slave configurations..
Wed Feb 1 20:16:44 2017 –
[info] read_only=1 is not set on slave
192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:16:44 2017 –
[warning] relay_log_purge=0 is not set
on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:16:44 2017 –
[info] Checking replication filtering settings..
Wed Feb 1 20:16:44 2017 –
[info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:16:44 2017 –
[info] Replication filtering check ok.
Wed Feb 1 20:16:44 2017 –
[info] GTID (with auto-pos) is not supported
Wed Feb 1 20:16:44 2017 –
[info] Starting SSH connection tests..
Wed Feb 1 20:16:50 2017 –
[info] All SSH connection tests passed successfully.
Wed Feb 1 20:16:50 2017 –
[info] Checking MHA Node version..
Wed Feb 1 20:16:51 2017 –
[info] Version check ok.
Wed Feb 1 20:16:51 2017 –
[info] Checking SSH publickey authentication settings on the current master..
Wed Feb 1 20:16:51 2017 –
[info] HealthCheck: SSH to 192.168.222.156 is reachable.
Wed Feb 1 20:16:51 2017 –
[info] Master MHA Node version is 0.57.
Wed Feb 1 20:16:51 2017 –
[info] Checking recovery script configurations on
192.168.222.156(192.168.222.156:3306)..
Wed Feb 1 20:16:51 2017 –
[info] Executing command:
save_binary_logs –command=test –start_pos=4 –binlog_dir=/home/binlog
–output_file=/tmp/save_binary_logs_test –manager_version=0.57
–start_file=mysql-bin.000058
Wed Feb 1 20:16:51 2017 –
[info] Connecting to
root@192.168.222.156(192.168.222.156:22)..
Failed to save binary log: Binlog not found from /home/binlog! If
you got this error at MHA Manager, please set
“master_binlog_dir=/path/to/binlog_directory_of_the_master” correctly
in the MHA Manager’s configuration file and try again.
at
/usr/local/bin/save_binary_logs line 123.
eval {…} called at
/usr/local/bin/save_binary_logs line 70
main::main() called at
/usr/local/bin/save_binary_logs line 66
Wed Feb 1 20:16:51 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln161] Binlog setting
check failed!
Wed Feb 1 20:16:51 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln408] Master
configuration failed.
Wed Feb 1 20:16:51 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/bin/masterha_check_repl line 48.
Wed Feb 1 20:16:51 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Wed Feb 1 20:16:51 2017 –
[info] Got exit code 1 (Not master dead).
处理：
修改 MHA配置文件中主库二进制日志的路径
[server default]
manager_log=/masterha/app1/app1.log
manager_workdir=/masterha/app1
#master_binlog_dir=/home/binlog 修改前的配置
master_binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data
master_ip_failover_script=/usr/local/bin/master_ip_failover
master_ip_online_change_script=/usr/local/bin/master_ip_online_change
password=123456
ping_interval=1
remote_workdir=/tmp
repl_password=123456
repl_user=repl
report_script=/usr/local/bin/send_report
secondary_check_script=/usr/local/bin/masterha_secondary_check -s
‘192.168.222.157’ -s ‘192.168.222.156’ –user=root
–master_ip=’192.168.222.156′ –master_port=3306
shutdown_script=””
ssh_user=root
user=root
[server1]
hostname=192.168.222.156
port=3306
[server2]
hostname=192.168.222.157
port=3306
candidate_master=1
check_repl_delay=0
问题14：Can’t exec “mysqlbinlog”: No such file or directory at
/usr/local/share/perl5/MHA/BinlogManager.pm line 106.
[root@mysqldb2 mysql]# masterha_check_repl
–conf=/etc/masterha/app1.cnf
Wed Feb 1 20:25:05 2017 –
[warning] Global configuration file /etc/masterha_default.cnf not found.
Skipping.
Wed Feb 1 20:25:05 2017 –
[info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:25:05 2017 –
[info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:25:05 2017 –
[info] MHA::MasterMonitor version 0.57.
Wed Feb 1 20:25:06 2017 –
[info] GTID failover mode = 0
Wed Feb 1 20:25:06 2017 –
[info] Dead Servers:
Wed Feb 1 20:25:06 2017 –
[info] Alive Servers:
Wed Feb 1 20:25:06 2017 –
[info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:25:06 2017 –
[info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:25:06 2017 –
[info] Alive Slaves:
Wed Feb 1 20:25:06 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Wed Feb 1 20:25:06 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:25:06 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Wed Feb 1 20:25:06 2017 –
[info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:25:06 2017 –
[info] Checking slave configurations..
Wed Feb 1 20:25:06 2017 –
[info] read_only=1 is not set on slave
192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:25:06 2017 –
[warning] relay_log_purge=0 is not set
on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:25:06 2017 –
[info] Checking replication filtering settings..
Wed Feb 1 20:25:06 2017 –
[info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:25:06 2017 –
[info] Replication filtering check ok.
Wed Feb 1 20:25:06 2017 –
[info] GTID (with auto-pos) is not supported
Wed Feb 1 20:25:06 2017 –
[info] Starting SSH connection tests..
Wed Feb 1 20:25:13 2017 –
[info] All SSH connection tests passed successfully.
Wed Feb 1 20:25:13 2017 –
[info] Checking MHA Node version..
Wed Feb 1 20:25:13 2017 –
[info] Version check ok.
Wed Feb 1 20:25:13 2017 –
[info] Checking SSH publickey authentication settings on the current master..
Wed Feb 1 20:25:13 2017 –
[info] HealthCheck: SSH to 192.168.222.156 is reachable.
Wed Feb 1 20:25:14 2017 –
[info] Master MHA Node version is 0.57.
Wed Feb 1 20:25:14 2017 –
[info] Checking recovery script configurations on
192.168.222.156(192.168.222.156:3306)..
Wed Feb 1 20:25:14 2017 –
[info] Executing command:
save_binary_logs –command=test –start_pos=4
–binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data –output_file=/tmp/save_binary_logs_test
–manager_version=0.57 –start_file=mysql-bin.000058
Wed Feb 1 20:25:14 2017 –
[info] Connecting to
root@192.168.222.156(192.168.222.156:22)..
Creating /tmp if not
exists.. ok.
Checking output directory is
accessible or not..
ok.
Binlog found at
/app/oracle/oradata2/mysql-5.5.37/data, up to mysql-bin.000058
Wed Feb 1 20:25:14 2017 –
[info] Binlog setting check done.
Wed Feb 1 20:25:14 2017 –
[info] Checking SSH publickey authentication and checking recovery script
configurations on all alive slave servers..
Wed Feb 1 20:25:14 2017 –
[info] Executing command :
apply_diff_relay_logs –command=test –slave_user=’root’
–slave_host=192.168.222.157 –slave_ip=192.168.222.157 –slave_port=3306
–workdir=/tmp –target_version=5.6.27-log –manager_version=0.57
–relay_log_info=/usr/local/mysql/data/relay-log.info –relay_dir=/usr/local/mysql/data/ –slave_pass=xxx
Wed Feb 1 20:25:14 2017 –
[info] Connecting to
root@192.168.222.157(192.168.222.157:22)..
Can’t exec “mysqlbinlog”: No such file or directory at
/usr/local/share/perl5/MHA/BinlogManager.pm line 106.
mysqlbinlog version command failed with rc 1:0, please verify PATH,
LD_LIBRARY_PATH, and client options
at
/usr/local/bin/apply_diff_relay_logs line 493.
Wed Feb 1 20:25:14 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln208] Slaves settings
check failed!
Wed Feb 1 20:25:14 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln416] Slave configuration
failed.
Wed Feb 1 20:25:14 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/bin/masterha_check_repl line 48.
Wed Feb 1 20:25:14 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Wed Feb 1 20:25:14 2017 –
[info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
处理：
[root@mysqldb2 mysql]# vi /etc/bashrc
#在文件的末尾添加
PATH=”$PATH:/usr/local/mysql/bin”
export PATH
问题15：Wed Feb 1 20:31:04 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking
configurations. Can’t exec “/usr/local/bin/master_ip_failover”: No
such file or directory at /usr/local/share/perl5/MHA/ManagerUtil.pm line 68.
[root@mysqldb2 mysql]# masterha_check_repl
–conf=/etc/masterha/app1.cnf
Wed Feb 1 20:30:56 2017 – [warning]
Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Feb 1 20:30:56 2017 –
[info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:30:56 2017 –
[info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:30:56 2017 –
[info] MHA::MasterMonitor version 0.57.
Wed Feb 1 20:30:57 2017 –
[info] GTID failover mode = 0
Wed Feb 1 20:30:57 2017 –
[info] Dead Servers:
Wed Feb 1 20:30:57 2017 –
[info] Alive Servers:
Wed Feb 1 20:30:57 2017 –
[info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:30:57 2017 –
[info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:30:57 2017 –
[info] Alive Slaves:
Wed Feb 1 20:30:57 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Wed Feb 1 20:30:57 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:30:57 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Wed Feb 1 20:30:57 2017 –
[info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:30:57 2017 –
[info] Checking slave configurations..
Wed Feb 1 20:30:57 2017 –
[info] read_only=1 is not set on slave
192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:30:57 2017 –
[warning] relay_log_purge=0 is not set
on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:30:57 2017 –
[info] Checking replication filtering settings..
Wed Feb 1 20:30:57 2017 –
[info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:30:57 2017 –
[info] Replication filtering check ok.
Wed Feb 1 20:30:57 2017 –
[info] GTID (with auto-pos) is not supported
Wed Feb 1 20:30:57 2017 –
[info] Starting SSH connection tests..
Wed Feb 1 20:31:03 2017 –
[info] All SSH connection tests passed successfully.
Wed Feb 1 20:31:03 2017 –
[info] Checking MHA Node version..
Wed Feb 1 20:31:03 2017 –
[info] Version check ok.
Wed Feb 1 20:31:03 2017 –
[info] Checking SSH publickey authentication settings on the current master..
Wed Feb 1 20:31:04 2017 –
[info] HealthCheck: SSH to 192.168.222.156 is reachable.
Wed Feb 1 20:31:04 2017 –
[info] Master MHA Node version is 0.57.
Wed Feb 1 20:31:04 2017 –
[info] Checking recovery script configurations on 192.168.222.156(192.168.222.156:3306)..
Wed Feb 1 20:31:04 2017 –
[info] Executing command:
save_binary_logs –command=test –start_pos=4
–binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data –output_file=/tmp/save_binary_logs_test
–manager_version=0.57 –start_file=mysql-bin.000058
Wed Feb 1 20:31:04 2017 –
[info] Connecting to
root@192.168.222.156(192.168.222.156:22)..
Creating /tmp if not
exists.. ok.
Checking output directory is
accessible or not..
ok.
Binlog found at
/app/oracle/oradata2/mysql-5.5.37/data, up to mysql-bin.000058
Wed Feb 1 20:31:04 2017 –
[info] Binlog setting check done.
Wed Feb 1 20:31:04 2017 –
[info] Checking SSH publickey authentication and checking recovery script
configurations on all alive slave servers..
Wed Feb 1 20:31:04 2017 –
[info] Executing command :
apply_diff_relay_logs –command=test –slave_user=’root’
–slave_host=192.168.222.157 –slave_ip=192.168.222.157 –slave_port=3306
–workdir=/tmp –target_version=5.6.27-log –manager_version=0.57 –relay_log_info=/usr/local/mysql/data/relay-log.info –relay_dir=/usr/local/mysql/data/ –slave_pass=xxx
Wed Feb 1 20:31:04 2017 –
[info] Connecting to
root@192.168.222.157(192.168.222.157:22)..
Checking slave recovery
environment settings..
Opening /usr/local/mysql/data/relay-log.info
… ok.
Relay log found at
/usr/local/mysql/data, up to mysqldb2-relay-bin.000005
Temporary relay log file
is /usr/local/mysql/data/mysqldb2-relay-bin.000005
Testing mysql connection
and privileges..Warning: Using a password on the command line interface can be
insecure.
done.
Testing mysqlbinlog
output.. done.
Cleaning up test file(s)..
done.
Wed Feb 1 20:31:04 2017 –
[info] Slaves settings check done.
Wed Feb 1 20:31:04 2017 –
[info]
192.168.222.156(192.168.222.156:3306) (current master)
+–192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:31:04 2017 –
[info] Checking replication health on 192.168.222.157..
Wed Feb 1 20:31:04 2017 –
[info] ok.
Wed Feb 1 20:31:04 2017 –
[info] Checking master_ip_failover_script status:
Wed Feb 1 20:31:04 2017 –
[info] /usr/local/bin/master_ip_failover –command=status –ssh_user=root
–orig_master_host=192.168.222.156 –orig_master_ip=192.168.222.156 –orig_master_port=3306
Wed Feb 1 20:31:04 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. Can’t exec
“/usr/local/bin/master_ip_failover”: No such file or directory at
/usr/local/share/perl5/MHA/ManagerUtil.pm line 68.
Wed Feb 1 20:31:04 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Wed Feb 1 20:31:04 2017 –
[info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
Wed Feb 1 20:31:04 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln229] Failed to get master_ip_failover_script
status with return code 1:0.
Wed Feb 1 20:31:04 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at /usr/local/bin/masterha_check_repl
line 48.
Wed Feb 1 20:31:04 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Wed Feb 1 20:31:04 2017 –
[info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
处理：
[root@mysqldb2 scripts]# cp master_ip_failover /usr/local/bin
[root@mysqldb2 scripts]# pwd
/root/olm/mha/mha4mysql-manager-master/samples/scripts
问题16：
[root@mysqldb2 scripts]# masterha_check_repl
–conf=/etc/masterha/app1.cnf
Wed Feb 1 20:40:41 2017 –
[warning] Global configuration file /etc/masterha_default.cnf not found.
Skipping.
Wed Feb 1 20:40:41 2017 –
[info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:40:41 2017 –
[info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:40:41 2017 –
[info] MHA::MasterMonitor version 0.57.
Wed Feb 1 20:40:43 2017 –
[info] GTID failover mode = 0
Wed Feb 1 20:40:43 2017 –
[info] Dead Servers:
Wed Feb 1 20:40:43 2017 –
[info] Alive Servers:
Wed Feb 1 20:40:43 2017 –
[info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:40:43 2017 –
[info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:40:43 2017 –
[info] Alive Slaves:
Wed Feb 1 20:40:43 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version
between slaves) log-bin:enabled
Wed Feb 1 20:40:43 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:40:43 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Wed Feb 1 20:40:43 2017 –
[info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:40:43 2017 –
[info] Checking slave configurations..
Wed Feb 1 20:40:43 2017 –
[info] read_only=1 is not set on slave
192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:40:43 2017 –
[warning] relay_log_purge=0 is not set
on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:40:43 2017 –
[info] Checking replication filtering settings..
Wed Feb 1 20:40:43 2017 –
[info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:40:43 2017 –
[info] Replication filtering check ok.
Wed Feb 1 20:40:43 2017 –
[info] GTID (with auto-pos) is not supported
Wed Feb 1 20:40:43 2017 –
[info] Starting SSH connection tests..
Wed Feb 1 20:40:59 2017 –
[info] All SSH connection tests passed successfully.
Wed Feb 1 20:40:59 2017 –
[info] Checking MHA Node version..
Wed Feb 1 20:40:59 2017 –
[info] Version check ok.
Wed Feb 1 20:40:59 2017 –
[info] Checking SSH publickey authentication settings on the current master..
Wed Feb 1 20:40:59 2017 –
[info] HealthCheck: SSH to 192.168.222.156 is reachable.
Wed Feb 1 20:40:59 2017 –
[info] Master MHA Node version is 0.57.
Wed Feb 1 20:40:59 2017 –
[info] Checking recovery script configurations on
192.168.222.156(192.168.222.156:3306)..
Wed Feb 1 20:40:59 2017 –
[info] Executing command:
save_binary_logs –command=test –start_pos=4
–binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data –output_file=/tmp/save_binary_logs_test
–manager_version=0.57 –start_file=mysql-bin.000058
Wed Feb 1 20:40:59 2017 –
[info] Connecting to
root@192.168.222.156(192.168.222.156:22)..
Creating /tmp if not
exists.. ok.
Checking output directory is
accessible or not..
ok.
Binlog found at
/app/oracle/oradata2/mysql-5.5.37/data, up to mysql-bin.000058
Wed Feb 1 20:41:00 2017 –
[info] Binlog setting check done.
Wed Feb 1 20:41:00 2017 –
[info] Checking SSH publickey authentication and checking recovery script
configurations on all alive slave servers..
Wed Feb 1 20:41:00 2017 –
[info] Executing command :
apply_diff_relay_logs –command=test –slave_user=’root’
–slave_host=192.168.222.157 –slave_ip=192.168.222.157 –slave_port=3306
–workdir=/tmp –target_version=5.6.27-log –manager_version=0.57
–relay_log_info=/usr/local/mysql/data/relay-log.info –relay_dir=/usr/local/mysql/data/ –slave_pass=xxx
Wed Feb 1 20:41:00 2017 –
[info] Connecting to
root@192.168.222.157(192.168.222.157:22)..
Checking slave recovery
environment settings..
Opening
/usr/local/mysql/data/relay-log.info … ok.
Relay log found at
/usr/local/mysql/data, up to mysqldb2-relay-bin.000005
Temporary relay log file
is /usr/local/mysql/data/mysqldb2-relay-bin.000005
Testing mysql connection
and privileges..Warning: Using a password on the command line interface can be
insecure.
done.
Testing mysqlbinlog
output.. done.
Cleaning up test file(s)..
done.
Wed Feb 1 20:41:00 2017 –
[info] Slaves settings check done.
Wed Feb 1 20:41:00 2017 –
[info]
192.168.222.156(192.168.222.156:3306) (current master)
+–192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:41:00 2017 –
[info] Checking replication health on 192.168.222.157..
Wed Feb 1 20:41:00 2017 –
[info] ok.
Wed Feb 1 20:41:00 2017 –
[info] Checking master_ip_failover_script status:
Wed Feb 1 20:41:00 2017 –
[info] /usr/local/bin/master_ip_failover –command=status –ssh_user=root
–orig_master_host=192.168.222.156 –orig_master_ip=192.168.222.156 –orig_master_port=3306
Bareword “FIXME_xxx” not allowed while “strict
subs” in use at /usr/local/bin/master_ip_failover line 93.
Execution of /usr/local/bin/master_ip_failover aborted due to
compilation errors.
Wed Feb 1 20:41:00 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln229] Failed to get master_ip_failover_script
status with return code 255:0.
Wed Feb 1 20:41:00 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/bin/masterha_check_repl line 48.
Wed Feb 1 20:41:00 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Wed Feb 1 20:41:00 2017 –
[info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
处理：
暂时注释 master_ip_failover 脚本
############################################################################################################################
Keepalived
############################################################################################################################
问题17：
./configure –prefix=/usr/local/keepalived
configure: error:
!!! OpenSSL is not properly
installed on your system. !!!
!!! Can not include OpenSSL
headers files. !!!
处理：
yum -y install openssl-devel
问题18：
[root@mysqldb2 keepalived-1.3.2]# make
Making all in lib
make[1]: Entering directory `/root/olm/mha/keepalived-1.3.2/lib’
make all-am
make[2]: Entering directory `/root/olm/mha/keepalived-1.3.2/lib’
CC memory.o
CC utils.o
CC notify.o
notify.c: 在函数‘find_path’中:
notify.c:226: 警告：此函数中的‘sgid_num’在使用前可能未初始化
CC timer.o
CC scheduler.o
CC vector.o
CC list.o
CC html.o
CC parser.o
CC signals.o
CC logger.o
CC rttables.o
AR liblib.a
make[2]: Leaving directory `/root/olm/mha/keepalived-1.3.2/lib’
make[1]: Leaving directory `/root/olm/mha/keepalived-1.3.2/lib’
Making all in keepalived
make[1]: Entering directory `/root/olm/mha/keepalived-1.3.2/keepalived’
Making all in core
make[2]: Entering directory
`/root/olm/mha/keepalived-1.3.2/keepalived/core’
CC main.o
CC daemon.o
CC pidfile.o
CC layer4.o
CC smtp.o
CC global_data.o
CC global_parser.o
CC process.o
CC namespaces.o
namespaces.c: 在函数‘setns’中:
namespaces.c:187: 错误：‘SYS_setns’未声明(在此函数内第一次使用)
namespaces.c:187: 错误：(即使在一个函数内多次出现，每个未声明的标识符在其
namespaces.c:187: 错误：所在的函数内也只报告一次。)
make[2]: *** [namespaces.o] 错误 1
make[2]: Leaving directory
`/root/olm/mha/keepalived-1.3.2/keepalived/core’
make[1]: *** [all-recursive] 错误 1
make[1]: Leaving directory
`/root/olm/mha/keepalived-1.3.2/keepalived’
make: *** [all-recursive] 错误 1
处理：
—安装 glibc 库仍无法解决问题
yum -y install glibc*
—使用低版本keepalived （keepalived-1.2.18）
问题19：
[root@oradb sysbench]# ./sysbench –test=olm
–oltp-table-size=1000000 –oltp-read-only=off –init-rng=on –num-threads=16
–max-requests=0 –oltp-dist-type=uniform –max-time=1800 –mysql-user=root
–db-driver=mysql –mysql-table-engine=innodb –oltp-test-mode=complexprepare
./sysbench: /usr/lib64/libmysqlclient.so.18: no version information
available (required by ./sysbench)
处理：
ln -s /app/oracle/oradata2/mysql-5.6.27/lib/libmysqlclient.so
/usr/lib64/libmysqlclient.so.18
问题：20
[root@mysqldb2 ~]# masterha_check_repl –conf=/etc/masterha/app1.cnf
Thu Feb 2 19:13:17 2017 –
[warning] Global configuration file /etc/masterha_default.cnf not found.
Skipping.
Thu Feb 2 19:13:17 2017 –
[info] Reading application default configuration from /etc/masterha/app1.cnf..
Thu Feb 2 19:13:17 2017 –
[info] Reading server configuration from /etc/masterha/app1.cnf..
Thu Feb 2 19:13:17 2017 –
[info] MHA::MasterMonitor version 0.57.
Thu Feb 2 19:13:17 2017 –
[error][/usr/local/share/perl5/MHA/Server.pm, ln180] Got MySQL error when
connecting 192.168.222.157(192.168.222.157:3306) :1045:Access denied for user
‘root’@’mysqldb2’ (using password: YES), but this is not a MySQL crash. Check
MySQL server settings.
Thu Feb 2 19:13:17 2017 –
[error][/usr/local/share/perl5/MHA/ServerManager.pm, ln301] at
/usr/local/share/perl5/MHA/ServerManager.pm line 297.
Thu Feb 2 19:13:18 2017 –
[error][/usr/local/share/perl5/MHA/ServerManager.pm, ln309] Got fatal error,
stopping operations
Thu Feb 2 19:13:18 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on
checking configurations. at
/usr/local/share/perl5/MHA/MasterMonitor.pm line 329.
Thu Feb 2 19:13:18 2017 –
[error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on
monitoring servers.
Thu Feb 2 19:13:18 2017 –
[info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
原因：从库没有MHA监控用户
处理：
1.再主库创建监控用户
2.然后导出DUMP文件
3.导入备库创建 主从复制
4.执行权限刷新 flush privileges;
问题21：
Thu Feb 2 19:56:56 2017 –
[info] Connecting to the target slave host 192.168.222.157, running recover
script..
Thu Feb 2 19:56:56 2017 –
[info] Executing command: apply_diff_relay_logs –command=apply
–slave_user=’root’ –slave_host=192.168.222.157 –slave_ip=192.168.222.157 –slave_port=3306
–apply_files=/tmp/saved_master_binlog_from_192.168.222.156_3306_20170202195650.binlog
–workdir=/tmp –target_version=5.6.27-log –timestamp=20170202195650
–handle_raw_binlog=1 –disable_log_bin=0 –manager_version=0.57
–slave_pass=xxx
Thu Feb 2 19:56:57 2017 –
[info]
MySQL client version is 5.6.27. Using –binary-mode.
Applying differential binary/relay log files
/tmp/saved_master_binlog_from_192.168.222.156_3306_20170202195650.binlog on
192.168.222.157:3306. This may take long time.
…..
Thu Feb 2 19:56:57 2017 –
[info]
MySQL client version is 5.6.27. Using –binary-mode.
Applying differential binary/relay log files
/tmp/saved_master_binlog_from_192.168.222.156_3306_20170202195650.binlog on
192.168.222.157:3306. This may take long time…
FATAL: applying log files failed with rc 1:0!
Error logs from
mysqldb2:/tmp/relay_log_apply_for_192.168.222.157_3306_20170202195650_err.log
(the last 200 lines)..
…..
at line 197: Cannot execute statement: impossible to write to binary
log since statement is in row format and BINLOG_FORMAT = STATEMENT
….
————–
Bye
at
/usr/local/bin/apply_diff_relay_logs line 515.
eval {…} called at
/usr/local/bin/apply_diff_relay_logs line 475
main::main() called at
/usr/local/bin/apply_diff_relay_logs line 120
Thu Feb 2 19:56:57 2017 –
[error][/usr/local/share/perl5/MHA/MasterFailover.pm, ln1398] Applying diffs failed with return code 22:0.
Thu Feb 2 19:56:57 2017 –
[error][/usr/local/share/perl5/MHA/MasterFailover.pm, ln1561] Recovering master
server failed.
Thu Feb 2 19:56:57 2017 –
[error][/usr/local/share/perl5/MHA/ManagerUtil.pm, ln177] Got ERROR: at /usr/local/bin/masterha_manager line 65.
Thu Feb 2 19:56:57 2017 –
[info]
—– Failover Report —–
app1: MySQL Master failover 192.168.222.156(192.168.222.156:3306)
Master 192.168.222.156(192.168.222.156:3306) is down!
Check MHA Manager logs at mysqldb2:/masterha/app1/app1.log for
details.
Started automated(non-interactive) failover.
Invalidated master IP address on
192.168.222.156(192.168.222.156:3306)
The latest slave 192.168.222.157(192.168.222.157:3306) has all relay
logs for recovery.
Selected 192.168.222.157(192.168.222.157:3306) as a new master.
Recovering master server failed.
Got Error so couldn’t continue failover from here.
原因：二进制日志格式错误，需要修改为行格式
处理：
1.修改主库和备库的二进制日志格式
vi my.cnf
#添加
binlog_format=row
重启主备库MYSQL
2. 启动mha监控
nohup masterha_manager –conf=/etc/masterha/app1.cnf
–remove_dead_master_conf –ignore_last_failover< /dev/null
>/masterha/app1/manager.log 2>&1 &
Thu Feb 2 20:24:51 2017 –
[info] MHA::MasterMonitor version 0.57.
Thu Feb 2 20:24:52 2017 –
[info] GTID failover mode = 0
Thu Feb 2 20:24:52 2017 –
[info] Dead Servers:
Thu Feb 2 20:24:52 2017 –
[info] Alive Servers:
Thu Feb 2 20:24:52 2017 –
[info] 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:24:52 2017 –
[info] 192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:24:52 2017 –
[info] Alive Slaves:
Thu Feb 2 20:24:52 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version
between slaves) log-bin:enabled
Thu Feb 2 20:24:52 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:24:52 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Thu Feb 2 20:24:52 2017 –
[info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:24:52 2017 –
[info] Checking slave configurations..
Thu Feb 2 20:24:52 2017 –
[info] read_only=1 is not set on slave
192.168.222.157(192.168.222.157:3306).
Thu Feb 2 20:24:52 2017 –
[warning] relay_log_purge=0 is not set
on slave 192.168.222.157(192.168.222.157:3306).
Thu Feb 2 20:24:52 2017 –
[info] Checking replication filtering settings..
Thu Feb 2 20:24:52 2017 –
[info] binlog_do_db= , binlog_ignore_db=
Thu Feb 2 20:24:52 2017 –
[info] Replication filtering check ok.
Thu Feb 2 20:24:52 2017 –
[info] GTID (with auto-pos) is not supported
Thu Feb 2 20:24:52 2017 –
[info] Starting SSH connection tests..
Thu Feb 2 20:24:54 2017 –
[info] All SSH connection tests passed successfully.
Thu Feb 2 20:24:54 2017 –
[info] Checking MHA Node version..
Thu Feb 2 20:24:54 2017 –
[info] Version check ok.
Thu Feb 2 20:24:54 2017 –
[info] Checking SSH publickey authentication settings on the current master..
Thu Feb 2 20:24:54 2017 –
[info] HealthCheck: SSH to 192.168.222.156 is reachable.
Thu Feb 2 20:24:55 2017 –
[info] Master MHA Node version is 0.57.
Thu Feb 2 20:24:55 2017 –
[info] Checking recovery script configurations on
192.168.222.156(192.168.222.156:3306)..
Thu Feb 2 20:24:55 2017 –
[info] Executing command:
save_binary_logs –command=test –start_pos=4
–binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data –output_file=/tmp/save_binary_logs_test
–manager_version=0.57 –start_file=mysql-bin.000064
Thu Feb 2 20:24:55 2017 –
[info] Connecting to
root@192.168.222.156(192.168.222.156:22)..
Creating /tmp if not
exists.. ok.
Checking output directory is
accessible or not..
ok.
Binlog found at
/app/oracle/oradata2/mysql-5.5.37/data, up to mysql-bin.000064
Thu Feb 2 20:24:55 2017 –
[info] Binlog setting check done.
Thu Feb 2 20:24:55 2017 –
[info] Checking SSH publickey authentication and checking recovery script
configurations on all alive slave servers..
Thu Feb 2 20:24:55 2017 –
[info] Executing command :
apply_diff_relay_logs –command=test –slave_user=’root’
–slave_host=192.168.222.157 –slave_ip=192.168.222.157 –slave_port=3306
–workdir=/tmp –target_version=5.6.27-log –manager_version=0.57
–relay_log_info=/usr/local/mysql/data/relay-log.info –relay_dir=/usr/local/mysql/data/ –slave_pass=xxx
Thu Feb 2 20:24:55 2017 –
[info] Connecting to
root@192.168.222.157(192.168.222.157:22)..
Checking slave recovery environment
settings..
Opening
/usr/local/mysql/data/relay-log.info … ok.
Relay log found at
/usr/local/mysql/data, up to relay-bin.000005
Temporary relay log file
is /usr/local/mysql/data/relay-bin.000005
Testing mysql connection
and privileges..Warning: Using a password on the command line interface can be
insecure.
done.
Testing mysqlbinlog
output.. done.
Cleaning up test file(s)..
done.
Thu Feb 2 20:24:56 2017 –
[info] Slaves settings check done.
Thu Feb 2 20:24:56 2017 – [info]
192.168.222.156(192.168.222.156:3306) (current master)
+–192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:24:56 2017 –
[info] Checking master_ip_failover_script status:
Thu Feb 2 20:24:56 2017 –
[info] /usr/local/bin/master_ip_failover –command=status –ssh_user=root
–orig_master_host=192.168.222.156 –orig_master_ip=192.168.222.156
–orig_master_port=3306
Thu Feb 2 20:24:56 2017 –
[info] OK.
Thu Feb 2 20:24:56 2017 –
[warning] shutdown_script is not defined.
Thu Feb 2 20:24:56 2017 – [info]
Set master ping interval 1 seconds.
Thu Feb 2 20:24:56 2017 –
[info] Set secondary check script: /usr/local/bin/masterha_secondary_check -s oradb –user=root –master_host=oradb
–master_ip=’192.168.222.156′ –master_port=3306
Thu Feb 2 20:24:56 2017 –
[info] Starting ping health check on 192.168.222.156(192.168.222.156:3306)..
Thu Feb 2 20:24:56 2017 –
[info] Ping(SELECT) succeeded, waiting until MySQL doesn’t respond..
3. 关闭主库
mysqladmin -u root shutdown
4. 启动MHA监控
nohup masterha_manager –conf=/etc/masterha/app1.cnf
–remove_dead_master_conf –ignore_last_failover< /dev/null
>/masterha/app1/manager.log 2>&1 &
Thu Feb 2 20:37:56 2017 –
[warning] Got error on MySQL select ping: 2013 (Lost connection to MySQL server
during query)
Thu Feb 2 20:37:56 2017 –
[info] Executing secondary network check script:
/usr/local/bin/masterha_secondary_check -s oradb –user=root –master_host=oradb –master_ip=’192.168.222.156′
–master_port=3306 –user=root –master_host=192.168.222.156 –master_ip=192.168.222.156 –master_port=3306 –master_user=root
–master_password=123456 –ping_type=SELECT
Thu Feb 2 20:37:56 2017 –
[info] Executing SSH check script: save_binary_logs –command=test
–start_pos=4 –binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data
–output_file=/tmp/save_binary_logs_test –manager_version=0.57
–binlog_prefix=mysql-bin
Thu Feb 2 20:37:57 2017 –
[warning] Got error on MySQL connect: 2003 (Can’t connect to MySQL server on
‘192.168.222.156’ (111))
Thu Feb 2 20:37:57 2017 –
[warning] Connection failed 2 time(s)..
Thu Feb 2 20:37:58 2017 –
[warning] Got error on MySQL connect: 2003 (Can’t connect to MySQL server on
‘192.168.222.156’ (111))
Thu Feb 2 20:37:58 2017 –
[warning] Connection failed 3 time(s)..
Thu Feb 2 20:37:59 2017 – [warning]
Got error on MySQL connect: 2003 (Can’t connect to MySQL server on
‘192.168.222.156’ (111))
Thu Feb 2 20:37:59 2017 –
[warning] Connection failed 4 time(s)..
Thu Feb 2 20:38:01 2017 –
[warning] HealthCheck: Got timeout on checking SSH connection to
192.168.222.156! at /usr/local/share/perl5/MHA/HealthCheck.pm line 342.
Monitoring server oradb is reachable, Master is not reachable from
oradb. OK.
Thu Feb 2 20:38:08 2017 –
[info] Master is not reachable from all other monitoring servers. Failover should
start.
Thu Feb 2 20:38:08 2017 –
[warning] Master is not reachable from health checker!
Thu Feb 2 20:38:08 2017 –
[warning] Master 192.168.222.156(192.168.222.156:3306) is not reachable!
Thu Feb 2 20:38:08 2017 –
[warning] SSH is NOT reachable.
Thu Feb 2 20:38:08 2017 –
[info] Connecting to a master server failed. Reading configuration file
/etc/masterha_default.cnf and /etc/masterha/app1.cnf again, and trying to
connect to all servers to check server status..
Thu Feb 2 20:38:08 2017 –
[warning] Global configuration file /etc/masterha_default.cnf not found.
Skipping.
Thu Feb 2 20:38:08 2017 –
[info] Reading application default configuration from /etc/masterha/app1.cnf..
Thu Feb 2 20:38:08 2017 –
[info] Reading server configuration from /etc/masterha/app1.cnf..
Thu Feb 2 20:38:09 2017 –
[warning] SQL Thread is stopped(no error) on
192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:38:09 2017 –
[info] GTID failover mode = 0
Thu Feb 2 20:38:09 2017 –
[info] Dead Servers:
Thu Feb 2 20:38:09 2017 –
[info] 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:09 2017 –
[info] Alive Servers:
Thu Feb 2 20:38:09 2017 –
[info] 192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:38:09 2017 –
[info] Alive Slaves:
Thu Feb 2 20:38:09 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version
between slaves) log-bin:enabled
Thu Feb 2 20:38:09 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:09 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Thu Feb 2 20:38:09 2017 –
[info] Checking slave configurations..
Thu Feb 2 20:38:09 2017 –
[info] read_only=1 is not set on slave
192.168.222.157(192.168.222.157:3306).
Thu Feb 2 20:38:09 2017 –
[warning] relay_log_purge=0 is not set
on slave 192.168.222.157(192.168.222.157:3306).
Thu Feb 2 20:38:09 2017 –
[info] Checking replication filtering settings..
Thu Feb 2 20:38:09 2017 –
[info] Replication filtering check ok.
Thu Feb 2 20:38:09 2017 –
[info] Master is down!
Thu Feb 2 20:38:09 2017 –
[info] Terminating monitoring script.
Thu Feb 2 20:38:09 2017 –
[info] Got exit code 20 (Master dead).
Thu Feb 2 20:38:09 2017 –
[info] MHA::MasterFailover version 0.57.
Thu Feb 2 20:38:09 2017 –
[info] Starting master failover.
Thu Feb 2 20:38:09 2017 –
[info]
Thu Feb 2 20:38:09 2017 –
[info] * Phase 1: Configuration Check Phase..
Thu Feb 2 20:38:09 2017 –
[info]
Thu Feb 2 20:38:10 2017 –
[warning] SQL Thread is stopped(no error) on 192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:38:10 2017 –
[info] GTID failover mode = 0
Thu Feb 2 20:38:10 2017 –
[info] Dead Servers:
Thu Feb 2 20:38:10 2017 –
[info] 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:10 2017 –
[info] Checking master reachability via MySQL(double check)…
Thu Feb 2 20:38:10 2017 –
[info] ok.
Thu Feb 2 20:38:10 2017 –
[info] Alive Servers:
Thu Feb 2 20:38:10 2017 –
[info] 192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:38:10 2017 –
[info] Alive Slaves:
Thu Feb 2 20:38:10 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Thu Feb 2 20:38:10 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:10 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Thu Feb 2 20:38:10 2017 –
[info] Starting SQL thread on
192.168.222.157(192.168.222.157:3306) ..
Thu Feb 2 20:38:10 2017 –
[info] done.
Thu Feb 2 20:38:10 2017 –
[info] Starting Non-GTID based failover.
Thu Feb 2 20:38:10 2017 –
[info]
Thu Feb 2 20:38:10 2017 –
[info] ** Phase 1: Configuration Check Phase completed.
Thu Feb 2 20:38:10 2017 –
[info]
Thu Feb 2 20:38:10 2017 –
[info] * Phase 2: Dead Master Shutdown Phase..
Thu Feb 2 20:38:10 2017 –
[info]
Thu Feb 2 20:38:10 2017 –
[info] Forcing shutdown so that applications never connect to the current
master..
Thu Feb 2 20:38:10 2017 –
[info] Executing master IP deactivation script:
Thu Feb 2 20:38:10 2017 –
[info] /usr/local/bin/master_ip_failover –orig_master_host=192.168.222.156
–orig_master_ip=192.168.222.156 –orig_master_port=3306 –command=stop
Thu Feb 2 20:38:10 2017 –
[info] done.
Thu Feb 2 20:38:10 2017 –
[warning] shutdown_script is not set. Skipping explicit shutting down of the
dead master.
Thu Feb 2 20:38:10 2017 –
[info] * Phase 2: Dead Master Shutdown Phase completed.
Thu Feb 2 20:38:10 2017 –
[info]
Thu Feb 2 20:38:10 2017 –
[info] * Phase 3: Master Recovery Phase..
Thu Feb 2 20:38:10 2017 –
[info]
Thu Feb 2 20:38:10 2017 –
[info] * Phase 3.1: Getting Latest Slaves Phase..
Thu Feb 2 20:38:10 2017 –
[info]
Thu Feb 2 20:38:10 2017 –
[info] The latest binary log file/position on all slaves is
mysql-bin.000064:120
Thu Feb 2 20:38:10 2017 –
[info] Latest slaves (Slaves that received relay log files to the latest):
Thu Feb 2 20:38:11 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Thu Feb 2 20:38:11 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:11 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Thu Feb 2 20:38:11 2017 –
[info] The oldest binary log file/position on all slaves is mysql-bin.000064:120
Thu Feb 2 20:38:11 2017 –
[info] Oldest slaves:
Thu Feb 2 20:38:11 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Thu Feb 2 20:38:11 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:11 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] * Phase 3.2: Saving Dead Master’s Binlog Phase..
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[warning] Dead Master is not SSH reachable. Could not save it’s binlogs.
Transactions that were not sent to the latest slave (Read_Master_Log_Pos to the
tail of the dead master’s binlog) were lost.
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] * Phase 3.3: Determining New Master Phase..
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] Finding the latest slave that has all relay logs for recovering other
slaves..
Thu Feb 2 20:38:11 2017 –
[info] All slaves received relay logs to the same position. No need to resync
each other.
Thu Feb 2 20:38:11 2017 –
[info] Searching new master from slaves..
Thu Feb 2 20:38:11 2017 –
[info] Candidate masters from the
configuration file:
Thu Feb 2 20:38:11 2017 –
[info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Thu Feb 2 20:38:11 2017 –
[info] Replicating from
192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:11 2017 –
[info] Primary candidate for the new
Master (candidate_master is set)
Thu Feb 2 20:38:11 2017 –
[info] Non-candidate masters:
Thu Feb 2 20:38:11 2017 –
[info] Searching from candidate_master
slaves which have received the latest relay log events..
Thu Feb 2 20:38:11 2017 –
[info] New master is 192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:38:11 2017 –
[info] Starting master failover..
Thu Feb 2 20:38:11 2017 –
[info]
From:
192.168.222.156(192.168.222.156:3306) (current master)
+–192.168.222.157(192.168.222.157:3306)
To:
192.168.222.157(192.168.222.157:3306) (new master)
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] * Phase 3.3: New Master Diff Log Generation Phase..
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] This server has all relay logs.
No need to generate diff files from the latest slave.
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] * Phase 3.4: Master Log Apply Phase..
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] *NOTICE: If any error happens from this phase, manual recovery is
needed.
Thu Feb 2 20:38:11 2017 –
[info] Starting recovery on 192.168.222.157(192.168.222.157:3306)..
Thu Feb 2 20:38:11 2017 –
[info] This server has all relay logs.
Waiting all logs to be applied..
Thu Feb 2 20:38:11 2017 –
[info] done.
Thu Feb 2 20:38:11 2017 –
[info] All relay logs were successfully
applied.
Thu Feb 2 20:38:11 2017 – [info]
Getting new master’s binlog name and position..
Thu Feb 2 20:38:11 2017 –
[info] mysql-bin.000003:120
Thu Feb 2 20:38:11 2017 –
[info] All other slaves should start
replication from here. Statement should be: CHANGE MASTER TO
MASTER_HOST=’192.168.222.157′, MASTER_PORT=3306,
MASTER_LOG_FILE=’mysql-bin.000003′, MASTER_LOG_POS=120, MASTER_USER=’repl’,
MASTER_PASSWORD=’xxx’;
Thu Feb 2 20:38:11 2017 –
[info] Executing master IP activate script:
Thu Feb 2 20:38:11 2017 –
[info] /usr/local/bin/master_ip_failover
–command=start –ssh_user=root –orig_master_host=192.168.222.156
–orig_master_ip=192.168.222.156 –orig_master_port=3306
–new_master_host=192.168.222.157 –new_master_ip=192.168.222.157
–new_master_port=3306 –new_master_user=’root’ –new_master_password=xxx
Undefined subroutine &main::FIXME_xxx_create_user called at
/usr/local/bin/master_ip_failover line 89.
Set read_only=0 on the new master.
Creating app user on the new master..
Thu Feb 2 20:38:11 2017 – [error][/usr/local/share/perl5/MHA/MasterFailover.pm,
ln1589] Failed to activate master IP
address for 192.168.222.157(192.168.222.157:3306) with return code 10:0
Thu Feb 2 20:38:11 2017 –
[warning] Proceeding.
Thu Feb 2 20:38:11 2017 –
[info] ** Finished master recovery successfully.
Thu Feb 2 20:38:11 2017 –
[info] * Phase 3: Master Recovery Phase completed.
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] * Phase 4: Slaves Recovery Phase..
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] * Phase 4.1: Starting Parallel Slave Diff Log Generation Phase..
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] Generating relay diff files from the latest slave succeeded.
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] * Phase 4.2: Starting Parallel Slave Log Apply Phase..
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] All new slave servers recovered successfully.
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] * Phase 5: New master cleanup phase..
Thu Feb 2 20:38:11 2017 –
[info]
Thu Feb 2 20:38:11 2017 –
[info] Resetting slave info on the new master..
Thu Feb 2 20:38:11 2017 –
[info] 192.168.222.157: Resetting slave
info succeeded.
Thu Feb 2 20:38:11 2017 –
[info] Master failover to 192.168.222.157(192.168.222.157:3306) completed
successfully.
Thu Feb 2 20:38:11 2017 –
[info] Deleted server1 entry from /etc/masterha/app1.cnf .
Thu Feb 2 20:38:11 2017 –
[info]
—– Failover Report —–
app1: MySQL Master failover 192.168.222.156(192.168.222.156:3306) to
192.168.222.157(192.168.222.157:3306) succeeded
Master 192.168.222.156(192.168.222.156:3306) is down!
Check MHA Manager logs at mysqldb2:/masterha/app1/app1.log for
details.
Started automated(non-interactive) failover.
Invalidated master IP address on
192.168.222.156(192.168.222.156:3306)
The latest slave 192.168.222.157(192.168.222.157:3306) has all relay
logs for recovery.
Selected 192.168.222.157(192.168.222.157:3306) as a new master.
192.168.222.157(192.168.222.157:3306): OK: Applying all logs
succeeded.
Failed to activate master IP address for
192.168.222.157(192.168.222.157:3306) with return code 10:0
Generating relay diff files from the latest slave succeeded.
192.168.222.157(192.168.222.157:3306): Resetting slave info
succeeded.
Master failover to 192.168.222.157(192.168.222.157:3306) completed
successfully.
