76.配置Impala负载均衡在Hue中

76.1 演示环境介绍

• CM和CDH版本：5.13.1
• Haproxy版本：1.5.18
• 已配置好Haproxy

76.2 操作演示

• Haproxy的配置：

global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    #option http-server-close
    #option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000
listen stats
    bind 0.0.0.0:1080
    mode http
    option httplog
    maxconn 5000
    stats refresh 30s
    stats  uri /stats 
listen hivejdbc
    bind 0.0.0.0:10099
    mode tcp
    option tcplog
    balance source
    server ip-186-31-5-190.fayson.com ip-186-31-5-190.fayson.com:10000 check
    server ip-186-31-6-148.fayson.com ip-186-31-6-148.fayson.com:10000 check
listen impalashell
    bind 0.0.0.0:25003
    mode tcp
    option tcplog
    balance leastconn
    server cdh04.fayson.com cdh04.fayson.com:21000 check
    server ip-186-31-5-190.fayson.com ip-186-31-5-190.fayson.com:21000 check
    server ip-186-31-10-118.fayson.com ip-186-31-10-118.fayson.com:21000 check
listen impalajdbc
    bind 0.0.0.0:25004
    mode tcp
    option tcplog
    balance source
    server cdh04.fayson.com cdh04.fayson.com:21050 check
    server ip-186-31-5-190.fayson.com ip-186-31-5-190.fayson.com:21050 check
    server ip-186-31-10-118.fayson.com ip-186-31-10-118.fayson.com:21050 check

• 修改Hue配置
  • 管理员登录CM的WEB控制台，进入hue服务的配置项页面
  • 在配置项搜索”hue_safety_valve.ini”，在该配置项增加如下配置
  • server_host: Haproxy服务的HOSTNAME或IP地址
  • server_port: Haproxy中配置监听的impalajdbc的端口25004

[impala]
server_host=ip-186-31-7-172.fayson.com
server_port=25004

• 保存配置，重启Hue及相关服务
  总结：
• 在Hue中配置连接多个ImpalaDaemon需要先配置Impala服务JDBC的负载均衡。
• 在配置impalajdbc的Haproxy时需要将负载均衡的策略指定为balance source否则在使用时会报“Results have expired, rerun the query ifneeded”
• balance source是将用户的IP地址经过Hash计算后制定到固定的服务器上

大数据视频推荐：
CSDN
大数据语音推荐：
企业级大数据技术应用
大数据机器学习案例之推荐系统
自然语言处理
大数据基础
人工智能：深度学习入门到精通