nginx+keepalived企业级web负载均衡架构（单主）

nginx+keepalived企业级web负载均衡架构

对于此企业级web负载均衡架构，参考自：http://network.51cto.com/art/201007/209823_all.htm
ps：主主集群 可以参考这个：http://www.php1.cn/article/71493.html

此系统架构仅映射内网VIP的80及443端口于外网的Juniper防火墙下，其它端口均关闭，内网所有机器均关闭iptables及ipfw防火墙；外网DNS指向即通过Juniper映射出来的外网地址，而此映射的地址对映的其实是内网VIP地址。

整个服务的架构就是这个样子↑↑↑↑↑↑↑↑

---

环境

• Centos 6.6
• JDK 1.6.0
• nginx 1.10.1
• tomcat 7.0
• keepalived 1.2.20
• 主nginx负载均衡器：192.168.1.241
• 辅nginx负载均衡器：192.168.1.243
• vip虚拟Ip：192.168.1.111

---

安装Nginx负载均衡器及相关配置

这个可以直接参考另一个笔记： 简单的nginx代理负载搭建
【ps：对于简单的搭建，↑这个笔记已经可以了，具体的nginx的配置 还是具体查百度吧。】

---

安装Keepalived，让其分别作web及Nginx的HA

安装keepalived，并将其做成服务模式，方便以后调试。

wget http://www.keepalived.org/software/keepalived-1.2.20.tar.gz
#tar zxvf keepalived-1.1.15.tar.gz  
#cd keepalived-1.1.15  
#./configure --prefix=/usr/local/keepalived  
#make   
#make install  
#cp /usr/local/keepalived/sbin/keepalived /usr/sbin/  
#cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/  
#cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/  
#mkdir /etc/keepalived  
#cd /etc/keepalived/  

vim keepalived.conf  
! Configuration File for keepalived  
global_defs {  
   notification_email {  
   1062767698@qq.com 
        }  
   notification_email_from keepalived@chtopnet.com  
   smtp_server 127.0.0.1  
   smtp_connect_timeout 30  
   router_id LVS_DEVEL  
}  
vrrp_instance VI_1 {  
    state MASTER  
    interface eth0  
    virtual_router_id 51  
    mcast_src_ip 192.168.1.241    <==主nginx的IP地址  
    priority 100  
    advert_int 1  
    authentication {  
        auth_type PASS  
        auth_pass chtopnet  
    }  
    virtual_ipaddress {  
        192.168.1.111                      <==vip地址  
    }  
}  
#service keepalived start 

辅机的配置文件：

! Configuration File for keepalived  
global_defs {  
   notification_email {  
   [email protected]  
        }  
   notification_email_from [email protected]  
   smtp_server 127.0.0.1  
   smtp_connect_timeout 30  
   router_id LVS_DEVEL  
}  
vrrp_instance VI_1 {  
    state BACKUP  
    interface eth0  
    virtual_router_id 51  
    mcast_src_ip 192.168.1.243              <==辅nginx的IP的地址  
    priority 99                             <==辅nginx的优先级 要小于主的
    advert_int 1  
    authentication {  
        auth_type PASS  
        auth_pass chtopnet  
    }  
    virtual_ipaddress {  
        192.168.1.111  
    }  
} 

开启服务，测试

---

开启两台keepalived的服务

【ps：因为是单主集群，所以要先启动主的keepalived，再启动辅的keepalived】

查看是否成功启动

查看日志：

[root@CentOS01 sbin]# tail /var/log/messages
Aug  9 19:26:22 CentOS01 Keepalived_healthcheckers[1323]: Using LinkWatch kernel netlink reflector...
Aug  9 19:26:26 CentOS01 Keepalived_vrrp[1324]: VRRP_Instance(VI_1) removing protocol VIPs.
Aug  9 19:26:26 CentOS01 Keepalived_vrrp[1324]: Using LinkWatch kernel netlink reflector...
Aug  9 19:26:26 CentOS01 Keepalived_vrrp[1324]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Aug  9 19:26:27 CentOS01 Keepalived_vrrp[1324]: VRRP_Instance(VI_1) Transition to MASTER STATE
Aug  9 19:26:28 CentOS01 Keepalived_vrrp[1324]: VRRP_Instance(VI_1) Entering MASTER STATE
Aug  9 19:26:28 CentOS01 Keepalived_vrrp[1324]: VRRP_Instance(VI_1) setting protocol VIPs.
Aug  9 19:26:28 CentOS01 Keepalived_vrrp[1324]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.111
Aug  9 19:26:28 CentOS01 Keepalived_healthcheckers[1323]: Netlink reflector reports IP 192.168.1.111 added
Aug  9 19:26:33 CentOS01 Keepalived_vrrp[1324]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.111

查看ip：

[root@CentOS01 sbin]# ip addr
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 00:0c:29:66:6a:49 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.241/24 brd 192.168.1.255 scope global eth0
    inet 192.168.1.111/32 scope global eth0
    inet6 fe80::20c:29ff:fe66:6a49/64 scope link
       valid_lft forever preferred_lft forever

以上已经说明vip已经启动了；

---

具体访问测试：

访问虚拟ip：192.168.1.111

↑↑↑↑↑↑↑此tomc为主nginx所配置的负载web服务器

---

关闭主nginx的keepalived服务，模拟此系统宕机；
【ps：生产上则需要脚本监控nginx运行情况开停止服务】
此时再访问虚拟ip：192.168.1.111：

---

自此，单主集群的服务搭建已经测试成功了。、
ps：单主，测试了发现，关闭辅nginx的keepalived是不会切换到主的nginx的。

---

脚本监控，完成真正意义的高可用

针对Nginx+Keepalived，编写nginx监控脚本nginx_pid.sh，此脚本思路其实也很简单，即放置在后台一直监控nginx进程；如进程消失，尝试重启nginx，如是失败则立即停掉本机的keepalived服务，让另一台负载均衡器接手，此脚本直接从生产环境下载：

vim /root/nginx_pid.sh   
#!/bin/bash  
while  :  
do  
 nginxpid=`ps -C nginx --no-header | wc -l`  
 if [ $nginxpid -eq 0 ];then  
  /usr/local/webserver/nginx/sbin/nginx  
  sleep 5  
  nginxpid=`ps -C nginx --no-header | wc -l`
   if [ $nginxpid -eq 0 ];then  
   /etc/init.d/keepalived stop  
   fi  
 fi  
 sleep 5   
done 

然后置于后台运行 sh /root/nginx_pid.sh &，这种写法是错误的，这样你用root用户logout后，此进程会消失；正确写法为nohup/bin/bash /root/nginx_pid.sh &,附带下注释:如果你正在运行一个进程，而且你觉得在退出帐户时该进程还不会结束，那么可以使用nohup命令。该命令可以在你退出root帐户之后继续运行相应的进程。nohup就是不挂起的意思( no hang up)

---

2016-08-09 星期二

---