Ambari集群搭建(CentOS7)

    本文以 ambari-2.6.2.2  、 HDP-2.6.5.0 、 HDP-UTILS-1.1.0.22 、HDP-GPL-2.6.5.0 为例搭建ambari集群。

官网地址：

ambari ：https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-installation/content/ambari_repositories.html

hdp：https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-installation/content/hdp_26_repositories.html

 

一、下载对应版本的镜像包

wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.2/ambari.repo
wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.2/ambari-2.6.2.2-centos7.tar.gz
wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/hdp.repo
wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos7-rpm.tar.gz
wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
wget http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/HDP-GPL-2.6.5.0-centos7-gpl.tar.gz

 

 

二、基础环境配置

1.修改主机名

2.修改映射

3.修改文件打开限制
[root@master ~]# vi /etc/security/limits.conf
# End of file
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

4.关闭防火墙
[root@master~]#systemctl disable firewalld
[root@master~]#systemctl stop firewalld

5.免密码登录

6.安装jdk8

 

 

三、安装httpd服务

yum install -y httpd
systemctl start httpd
systemctl enable httpd

 

四、配置yum源

# 创建文件夹
mkdir -p /var/www/html/ambari

# 解压至文件夹
tar -zxvf ambari-2.6.2.2-centos7.tar.gz -C /var/www/html/ambari
tar -zxvf HDP-2.6.5.0-centos7-rpm.tar.gz -C /var/www/html/ambari
tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/ambari
tar -zxvf HDP-GPL-2.6.5.0-centos7-gpl.tar.gz -C /var/www/html/ambari

# 复制ambari.repo、hdp.repo到yum
cp ambari.repo /etc/yum.repos.d/
cp hdp.repo /etc/yum.repos.d/

# 修改ambari.repo(ip为httpd主机ip)
################################################################
[ambari-2.6.2.2]
name=ambari Version - ambari-2.6.2.2
baseurl=http://192.168.2.101/ambari/ambari/centos7/2.6.2.2-1/
gpgcheck=1
gpgkey=http://192.168.2.101/ambari/ambari/centos7/2.6.2.2-1/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
################################################################


# 修改hdp.repo(ip为httpd主机ip)
################################################################
[HDP-2.6.5.0]
name=HDP Version - HDP-2.6.5.0
baseurl=http://192.168.2.101/ambari/HDP/centos7/2.6.5.0-292/
gpgcheck=1
gpgkey=http://192.168.2.101/ambari/HDP/centos7/2.6.5.0-292/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1


[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://192.168.2.101/ambari/HDP-UTILS/centos7/1.1.0.22/
gpgcheck=1
gpgkey=http://192.168.2.101/ambari/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
################################################################

# 将repo复制到其他节点(所有节点)
scp /etc/yum.repos.d/ambari.repo  server2:/etc/yum.repos.d/
scp /etc/yum.repos.d/hdp.repo     server2:/etc/yum.repos.d/

# 更新yum(所有节点)
yum clean all
yum makecache
yum update

 

 

五、安装ntp时间同步服务器

• 服务端

# 安装
yum -y install ntp

# 配置
vi /etc/ntp.conf
##############################################################
//将这行给注释
#restrict default nomodify notrap nopeer noquery      
 
//然后添加这两行 ，意思是 允许IPV4 IPV6 查询 拒绝修改
restrict -4 default kod notrap nomodify    
restrict -6 default kod notrap nomodify
 
//添加修改上层时间服务器，如果能联通外网，则不需要修改，
server 0.rhel.pool.ntp.org iburst
server 1.rhel.pool.ntp.org iburst
server 2.rhel.pool.ntp.org iburst
server 3.rhel.pool.ntp.org iburst
//如果联不通外网，则把上面几行注释，然后添加下面两行，意思是设置成时间服务器是自己
server 127.127.1.0
fudge 127.127.1.0 stratum 8
##############################################################

# 启动
systemctl  start  ntpd

• 客户端

# 安装
yum -y install ntp

# 加入定时器
crontab -e
#############################################
0 * * * * /usr/sbin/ntpdate 192.168.2.101
#############################################




或者
vim /etc/ntp.conf
#############################################
server master主机ip  iburst
#############################################

 

六、安装mysql5.7

• 安装（不能装mysql8.X）

             https://blog.csdn.net/qq_23160237/article/details/102467487

• 创建用户以及赋权

# 登录
mysql -uroot -p

# 赋权
create database ambari character set utf8;
Create user 'ambari' identified by 'ambari';
grant all privileges on *.* to 'ambari';
Flush privileges;

create database hive character set utf8;
create user 'hive' identified by 'hive';
grant all privileges on *.* to 'hive';
flush privileges;

 

七、安装ambari

yum install ambari-server

 

八、导入初始化数据

                使用ambari用户登录：mysql  -uambari -p

                

 

九、导入Mysql驱动

yum install mysql-connector-java
cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/mysql-connector-java.jar
ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar
vi /etc/ambari-server/conf/ambari.properties
添加server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar

或者

mkdir /usr/share/java
cp mysql-connector-java-5.1.48.jar /usr/share/java/mysql-connector-java.jar
cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/
vi /etc/ambari-server/conf/ambari.properties
添加server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar

 

十、配置ambari

[root@master ~]# ambari-server setup
下面是配置执行流程，按照提示操作
（1） 提示是否自定义设置。输入：y
Customize user account for ambari-server daemon [y/n] (n)? y

（2）ambari-server 账号。输入：root
Enter user account for ambari-server daemon (root):root
Adjusting ambari-server permissions and ownership...

（3）检查防火墙是否关闭。输入：y
Adjusting ambari-server permissions and ownership...
Checking firewall...
WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.
OK to continue [y/n] (y)? y

（4）设置JDK。输入：3
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /opt/jdk8
Validating JDK on Ambari Server...done.
Completing setup...

（5）数据库配置。选择：y
Configuring database...
Enter advanced database configuration [y/n] (n)? y

（6）选择数据库类型。输入：3
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
==============================================================================
Enter choice (3): 3

（7）设置数据库的具体配置信息，根据实际情况输入，如果和括号内相同，则可以直接回车。如果想重命名，就输入。
Hostname (localhost):localhost
Port (3306):3306
Database name (ambari):ambari
Username (ambari):ambari
Enter Database Password (bigdata):ambari
Re-Enter password: ambari

（8）是否远程连接。输入：y
Proceed with configuring remote database connection properties [y/n] (y)?  

 

 

十一、启动服务

ambari-server start

 

十二、服务安装

1）登录

              登录界面，默认管理员账户登录， 账户：admin 密码：admin

 

2）配置集群的名字为hadoop

 

3）选择版本并修改为本地源地址

 

 

（3）安装配置

 

 

（4）安装ambari的agent，同时检查系统问题

 

 

（5）选择要安装的服务

 

（6）选择分配服务

 

 

 

十三、错误处理

     如果以上步骤出现错误，查看日志（/var/log/ambari-server/ambari-server.log）

     或者根据以下步骤重置重新安装

[root@master ~]# ambari-server stop
[root@master ~]# ambari-server reset
[root@master ~]# ambari-server setup

[root@master ~]# mysql -uroot -p
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| ambari             |
| hive               |
| oozie              |
| performance_schema |
+--------------------+
rows in set (0.00 sec)
mysql> drop database ambari;
mysql> drop database hive;

 

常见报错：

    1.  hbase启动后，运行报错：

            Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /atsv2-hbase-              unsecure/hbaseid

            修改：ZooKeeper Znode Parent 的value为正确路径

    2.  hbase启动后，运行报错：

            ServerNotRunningYetException: Server is not running yet

            解决方案：

                 1）hdfs退出安全模式：sudo -u hdfs hadoop dfsadmin -safemode leave 

    3..hbase启动后，运行报错：

             PleaseHoldException: Master is initializing

             解决方案：

                 1）HBase root directory ： 修改为hdfs路径（hdfs://namenode-ha:8020/hbase）

 

卸载ambari

rm -rf /var/cache/yum/x86_64/7/*
rm -rf /usr/share/doc/smartsense-*
yum remove smartsense-hst
rm -rf /var/log/smartsense/