42.Impala负载均衡实现—HAProxy

42.1 操作演示

HAProxy安装及启停

• 在集群中选择一个节点，使用yum方式安装HAProxy服务

[root@ip-172-31-9-33 ~]# yum -y install haproxy

• 启动与停止HAProxy服务，并将服务添加到自启动列表

[root@ip-172-31-9-33 ~]# service haproxy start
[root@ip-172-31-9-33 ~]# service haproxy stop
[root@ip-172-31-9-33 ~]# chkconfig haproxy on

HAProxy配置Impala负载均衡

• 将/etc/haproxy目录下的haproxy.cfg文件备份，新建haproxy.cfg文件，添加如下配置
  • 主要配置了HAProxy的http状态管理界面、impalashell和impalajdbc的负载均衡。

#---------------------------------------------------------------------
# Example configuration for a possible web application.  See the
# full configuration options online.
#
#   http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    log         127.0.0.1 local2

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    #option http-server-close
    #option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000


listen stats
    bind 0.0.0.0:1080
    mode http
    option httplog
    maxconn 5000
    stats refresh 30s
    stats  uri /stats 

listen impalashell
    bind 0.0.0.0:25003
    mode tcp
    option tcplog
    balance leastconn
    server ip-172-31-9-33.fayson.com ip-172-31-9-33.fayson.com:21000 check
    server ip-172-31-5-190.fayson.com ip-172-31-5-190.fayson.com:21000 check
    server ip-172-31-10-118.fayson.com ip-172-31-10-118.fayson.com:21000 check

listen impalajdbc
    bind 0.0.0.0:25004
    mode tcp
    option tcplog
    balance leastconn
    server ip-172-31-9-33.fayson.com ip-172-31-9-33.fayson.com:21050 check
    server ip-172-31-5-190.fayson.com ip-172-31-5-190.fayson.com:21050 check
    server ip-172-31-10-118.fayson.com ip-172-31-10-118.fayson.com:21050 check

• 重启HAProxy服务

[root@ip-172-31-9-33 haproxy]# service haproxy restart

• 浏览器访问http://{hostname}:1080/stats，查看Impala服务的负载均衡是否配置成功

Impala Shell测试

• 使用多个终端同时访问，并执行SQL语句，查看是否会通过HAProxy服务自动负载到其它Impala Daemon节点
• 使用Impala shell访问HAProxy服务的25003端口，命令如下

[root@ip-172-31-6-148 ~]# impala-shell -i ip-172-31-9-33.fayson.com:25003

• 打开第一个终端访问并执行SQL

[root@ip-172-31-6-148 ~]# impala-shell -i ip-172-31-9-33.fayson.com:25003
...
[ip-172-31-9-33.fayson.com:25003] > select * from my_first_table;
...
+----+------+
| id | name |
+----+------+
| 1  | john |
| 2  | tom  |
| 3  | jim  |
+----+------+
Fetched 3 row(s) in 7.20s
[ip-172-31-9-33.fayson.com:25003] >

• 同时打开第二个终端访问并执行SQL

[root@ip-172-31-6-148 ~]# impala-shell -i ip-172-31-9-33.fayson.com:25003
...
[ip-172-31-9-33.fayson.com:25003] > select * from my_first_table;
...
+----+------+
| id | name |
+----+------+
| 2  | tom  |
| 3  | jim  |
| 1  | john |
+----+------+
Fetched 3 row(s) in 1.81s
[ip-172-31-9-33.fayson.com:25003] > 

• 通过以上测试可以看到，两个终端执行的SQL不在同一个Impala Daemon，这样就实现了Impala Daemon服务的负载均衡。

大数据视频推荐：
CSDN