在云端上HAproxy的每个实例将其前端配置到虚拟IP地址上，根据修改haroxy配置文件，使得Haproxy后端是用于负载平衡的真实实例IP地址的列表。通过使用负载均衡器，在发生单点故障的时候，可以快速的切换到其他的节点。 1）安装Haproxy 在三个节点上分别安装HAProxy ``` yum -y install haproxy systemctl enable haproxy.service ``` 2）跟rsyslog结合配置haproxy日志，在三个节点上都操作 ``` cd /etc/rsyslog.d/ vim haproxy.conf ``` 添加： ``` $ModLoad imudp $UDPServerRun 514 $template Haproxy,"%rawmsg%\n" local0.=info -/var/log/haproxy.log;Haproxy local0.notice -/var/log/haproxy-status.log;Haproxy local0.* ~ ``` ``` systemctl restart rsyslog.service systemctl enable rsyslog.service systemctl status rsyslog.service ``` 3）在是三个节点上配置haproxy.cfg ``` cd /etc/haproxy/ mv haproxy.cfg haproxy.cfg.orig vim haproxy.cfg ``` 添加下面内容（**注意更改ip**）： ``` global log 127.0.0.1 local0 log 127.0.0.1 local1 notice maxconn 16000 chroot /usr/share/haproxy user haproxy group haproxy daemon defaults log global mode http option tcplog option dontlognull retries 3 option redispatch maxconn 10000 contimeout 5000 clitimeout 50000 srvtimeout 50000 frontend stats-front bind *:8088 mode http default_backend stats-back backend stats-back mode http balance source stats uri /haproxy/stats stats auth admin:yjscloud listen RabbitMQ-Server-Cluster bind 192.168.0.168:56720 mode tcp balance roundrobin option tcpka server controller1 controller1:5672 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller2 controller2:5672 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller3 controller3:5672 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 listen RabbitMQ-Web bind 192.168.0.168:15673 mode tcp balance roundrobin option tcpka server controller1 controller1:15672 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller2 controller2:15672 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller3 controller3:15672 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 listen Galera-Cluster bind 192.168.0.168:3306 balance leastconn mode tcp option tcplog option httpchk server controller1 controller1:3306 check port 9200 inter 20s fastinter 2s downinter 2s rise 3 fall 3 server controller2 controller2:3306 check port 9200 inter 20s fastinter 2s downinter 2s rise 3 fall 3 server controller3 controller3:3306 check port 9200 inter 20s fastinter 2s downinter 2s rise 3 fall 3 listen keystone_admin_cluster bind 192.168.0.168:35357 balance source option httpchk option httplog option httpclose server controller1 controller1:35358 check inter 10s fastinter 2s downinter 2s rise 30 fall 3 server controller2 controller2:35358 check inter 10s fastinter 2s downinter 2s rise 30 fall 3 server controller3 controller3:35358 check inter 10s fastinter 2s downinter 2s rise 30 fall 3 listen keystone_public_internal_cluster bind 192.168.0.168:5000 balance source option httpchk option httplog option httpclose server controller1 controller1:5002 check inter 10s fastinter 2s downinter 2s rise 30 fall 3 server controller2 controller2:5002 check inter 10s fastinter 2s downinter 2s rise 30 fall 3 server controller3 controller3:5002 check inter 10s fastinter 2s downinter 2s rise 30 fall 3 listen Memcache_Servers bind 192.168.0.168:22122 balance roundrobin mode tcp option tcpka server controller1 controller1:11211 check inter 10s fastinter 2s downinter 2s rise 30 fall 3 server controller2 controller2:11211 check inter 10s fastinter 2s downinter 2s rise 30 fall 3 server controller3 controller3:11211 check inter 10s fastinter 2s downinter 2s rise 30 fall 3 listen dashboard_cluster bind 192.168.0.168:80 balance source option httpchk option httplog option httpclose server controller1 controller1:8080 check inter 2000 fall 3 server controller2 controller2:8080 check inter 2000 fall 3 server controller3 controller3:8080 check inter 2000 fall 3 listen glance_api_cluster bind 192.168.0.168:9292 balance source option httpchk option httplog option httpclose server controller1 controller1:9393 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller2 controller2:9393 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller3 controller3:9393 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 listen glance_registry_cluster bind 192.168.0.168:9090 balance roundrobin mode tcp option tcpka server controller1 controller1:9191 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller2 controller2:9191 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller3 controller3:9191 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 listen nova_compute_api_cluster bind 192.168.0.168:8774 balance source option httpchk option httplog option httpclose server controller1 controller1:9774 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller2 controller2:9774 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller3 controller3:9774 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 listen nova-metadata-api_cluster bind 192.168.0.168:8775 balance source option httpchk option httplog option httpclose server controller1 controller1:9775 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller2 controller2:9775 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller3 controller3:9775 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 listen nova_vncproxy_cluster bind 192.168.0.168:6080 balance source option tcpka option tcplog server controller1 controller1:6080 check inter 2000 rise 2 fall 5 server controller2 controller2:6080 check inter 2000 rise 2 fall 5 server controller3 controller3:6080 check inter 2000 rise 2 fall 5 listen neutron_api_cluster bind 192.168.0.168:9696 balance source option httpchk option httplog option httpclose server controller1 controller1:9797 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller2 controller2:9797 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller3 controller3:9797 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 listen cinder_api_cluster bind 192.168.0.168:8776 balance source option httpchk option httplog option httpclose server controller1 controller1:8778 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller2 controller2:8778 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 server controller3 controller3:8778 check inter 10s fastinter 2s downinter 3s rise 3 fall 3 ``` 将配置文件拷贝到其他两个节点并重启haproxy服务 ``` scp -p /etc/haproxy/haproxy.cfg controller2:/etc/haproxy/haproxy.cfg scp -p /etc/haproxy/haproxy.cfg controller3:/etc/haproxy/haproxy.cfg systemctl restart haproxy.service systemctl status haproxy.service ``` 参数解释： - inter <delay>：设定健康状态检查的时间间隔，单位为毫秒，默认为2000；也可以使用fastinter和downinter来根据服务器端状态优化此时间延迟 - rise <count>：设定健康状态检查中，某离线的server从离线状态转换至正常状态需要成功检查的次数。 - fall <count>：确认server从正常状态转换为不可用状态需要检查的次数。 4）配置Haproxy能监控Galera数据库集群 数据库如果无法正常启动请重启三个节点的mysql 其他两个节点分别启动mariadb ``` systemctl start mariadb.service systemctl status mariadb.service ``` 最后，其他两个节点启动成功了，再回到第一个节点执行： ``` pkill -9 mysql pkill -9 mysql systemctl start mariadb.service systemctl status mariadb.service ``` 在controller1上进入mysql，创建clustercheck ``` grant process on *.* to 'clustercheckuser'@'localhost' identified by 'clustercheckpassword!'; flush privileges; ``` 三个节点分别创建clustercheck文本，里面是clustercheckuser用户和密码 ``` vim /etc/sysconfig/clustercheck ``` 添加： ``` MYSQL_USERNAME=clustercheckuser MYSQL_PASSWORD=clustercheckpassword! MYSQL_HOST=localhost MYSQL_PORT=3306 ``` 确认下是否存在`/usr/bin/clustercheck`脚本，如果没有从网上下载一个，然后放到`/usr/bin`目录下面，记得`chmod +x /usr/bin/clustercheck`赋予权限 这个脚本的作用就是让haproxy能监控Galera cluster状态 ``` scp -p /usr/bin/clustercheck controller2:/usr/bin/clustercheck scp -p /usr/bin/clustercheck controller3:/usr/bin/clustercheck ``` 在controller1上检查haproxy服务状态 ``` clustercheck ```  结合xinetd监控Galera服务（三个节点都安装xinetd） ``` yum -y install xinetd vim /etc/xinetd.d/mysqlchk ``` 添加以下内容： ``` # default: on # # description: mysqlchk service mysqlchk { # this is a config for xinetd, place it in /etc/xinetd.d/ disable = no flags = REUSE socket_type = stream port = 9200 wait = no user = nobody server = /usr/bin/clustercheck log_on_failure += USERID only_from = 0.0.0.0/0 # recommended to put the IPs that need # to connect exclusively (security purposes) per_source = UNLIMITED } ``` 拷贝配置文件到其他节点 ``` scp -p /etc/xinetd.d/mysqlchk controller2:/etc/xinetd.d/mysqlchk scp -p /etc/xinetd.d/mysqlchk controller3:/etc/xinetd.d/mysqlchk ``` ``` vim /etc/services ``` 最后一行添加：`mysqlchk              9200/tcp #mysqlchk` ``` scp -p /etc/services controller2:/etc/services scp -p /etc/services controller3:/etc/services ``` 重启xinetd服务 ``` systemctl enable xinetd.service systemctl restart xinetd.service systemctl status xinetd.service ``` 5）三个节点修改内网参数 ``` echo 'net.ipv4.ip_nonlocal_bind = 1'>>/etc/sysctl.conf echo 'net.ipv4.ip_forward = 1'>>/etc/sysctl.conf sysctl -p ``` 第一个参数的意思是设置haproxy能够绑定到不属于本地网卡的地址上 第二个参数的意思是设置内核是否转发数据包，默认是禁止的，这里我们设置打开。 注意！如果不设置这两个参数，你的第二个第三个ctr节点haproxy服务将启动不了 6）三节点启动haproxy服务 ``` systemctl restart haproxy.service systemctl status haproxy.service ``` 7）访问haproxy前端web平台 ``` http://192.168.0.168:8088/haproxy/stats admin/yjscloud ``` Galera集群服务已经监控成功