本地部署Hive集群

规划

服务	机器
Hive本体	部署在Node1
元数据服务所需的关系型数据库(MYSQL)	部署在Node1

安装MYSQL数据库

# 更新密钥
rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022

# 安装Mysql yum库
rpm -Uvh http://repo.mysql.com//mysql57-community-release-el7-7.noarch.rpm

# yum安装MysqL
yum -y install mysql-community-server

# 启动MySQL并跳过授权表，使用--user=mysql选项来指定MySQL以mysql用户身份运行
mysqld --skip-grant-tables --skip-networking --user=mysql &

#  使用mysql登录并重置密码
mysql -uroot
USE mysql;
set global validate_password_policy=LOW;
set global validate_password_length=4;
ALTER USER 'root'@'localhost' IDENTIFIED BY '123456';
grant all privileges on *.* to root@"%" identified by 'root' with grant option;
FLUSH PRIVILEGES;

# 重启mysql并重新登录
systemctl stop mysqld
systemctl start mysqld
systemctl enable mysqld

mysql -uroot -p

配置Hadoop

 Hive的运行依赖于Hadoop(HDFS、Mapreduce、YARN都依赖)
同时涉及到hDFS文件系统的访问，所以需要配置Hadoop的代理用户
即设置hadoop用户允许代理(模拟)其他用户

配置如下内容在Hadoop的core-site.xml中，并分发到其他节点，且重启HDFS集群

  
    hadoop.proxyuser.hadoop.hosts
    *
  
  
    hadoop.proxyuser.hadoop.groups
    *
  

 下载解压Hive

hive和mysql驱动安装包位置：

apache-hive-3.1.3-bin.tar.gzhttps://archive.apache.org/dist/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gzmysql-connector-java-5.1.34.jarhttps://repo1.maven.org/maven2/mysql/mysql-connector-java/5.1.34/mysql-connector-java-5.1.34.jar

tar -zxf apache-hive-3.1.3-bin.tar.gz -C /export/server/

mv mysql-connector-java-5.1.34.jar  /export/server/apache-hive-3.1.3-bin/lib/

ln -s /export/server/apache-hive-3.1.3-bin hive

配置Hive

在Hive的conf目录内，新建hive-env.sh文件，填入以下环境变量内容：

export HADOOP_HOME=/export/server/hadoop
export HIVE_CONF_DIR=/export/server/hive/conf
export HIVE_AUX_JARS_PATH=/export/server/hive/lib

在Hive的conf目录内，新建hive-site.xml文件，填入以下内容：

vim hive-site.xml




	javax.jdo.option.ConnectionURL
	jdbc:mysql://node1:3306/hive?createDatabaseIfNotExist=true&useSSL=false&useUnicode=true&characterEncoding=UTF-8



	javax.jdo.option.ConnectionDriverName
	com.mysql.jdbc.Driver



	javax.jdo.option.ConnectionUserName
	root



	javax.jdo.option.ConnectionPassword
	hadoop




    hive.server2.thrift.bind.host
    node1


 # 远程模式，metastore必须单独配置

    hive.metastore.uris
    thrift://node1:9083


 

    hive.metastore.event.db.notification.api.auth
    false

初始化元数据库

mysql -uroot -p123456
create database hive charset utf8;
exit;

cd /export/server/hive/bin
./schematool -initSchema -dbType mysql -verbos

启动Hive(使用Hadoop用户)

chown -R hadoop:hadoop apache-hive-3.1.3-bin hive
cd /export/server/hive/
mkdir logs
nohup bin/hive --service metastore  >> logs/metastore.log 2>&1 &
cd ../bin
./hive