hadoop客户端安装 windows

1.把集群中的hadoop安装目录打包,解压到windows目录下

2.修改/hadoop/etc/haoop 下的配置文件

 core-site.xml


        
                fs.defaultFS
                hdfs://master:8020
        
        
                hadoop.tmp.dir
                D:/QEL/BD_client/hadoop/tmp
                Abase for other temporary directories.
        

 hdfs-site.xml


        
                dfs.namenode.secondary.http-address
                master:50090
        
        
                dfs.replication
                1
        
        
                dfs.namenode.name.dir
                D:/QEL/BD_client/hadoop/tmp/dfs/name
        
        
                dfs.datanode.data.dir
                D:/QEL/BD_client/hadoop/tmp/dfs/data
        

 hadoop-env.cmd

set JAVA_HOME=D:\QEL\Java\jdk1.8.0_181

3.修改windows环境变量 

新建HADOOP_HOME

然后将%HADOOP_HOME%\bin加入Path

4.配置本地用户变量,冒充hadoop用户

用户名为集群中的root用户

hadoop客户端安装 windows_第1张图片

5.下载到的hadooponwindows-master.zip,解压,将bin目录(包含以下.dll和.exe文件)文件替换原来hadoop目录下的bin目录

6.修改windows   host文件

路径C:\Windows\System32\drivers\etc

修改hosts,最后一行加入集群master的ip
192.168.1.100	master

 之后ping master测试

7.测试

cmd下路径改为hadoop下bin目录
hdfs dfs -mkdir hdfs://master:8020/windowsMake_master

7.问题

hdfs dfs -put C:\Users\lenovo\Desktop\put.txt hdfs://master:8020/windowsMake


报错
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File 
/windowsMake/put.txt._COPYING_ could only be replicated to 0 nodes instead of 
minReplication (=1).  There are 1 datanode(s) running and 1 node(s) are excluded in this 
operation.

当时按照别人windows集群操作的hadoop,对namenode 进行了格式化,将dfs.namenode.name.dir和dfs.datanode.data.di路径对应的文件删除即可!!!!

你可能感兴趣的:(大数据)