hadoop-0.20.1-examples.jar wordcount 例子运行出现的问题记录

hadoop版本 hadoop-0.20.2

执行一个wordcount的mapreduce作业

1.启动hadoop

$ cd hadoop-0.20.2/bin

bin$ hadoop namenode -format

bin$ start-all.sh

bin$ jps

bin$ hadoop dfsadmin -safemode leave

2.生成数据

bin$ hadoop fs -mkdir input

bin$ hadoop fs -put /home/dic/input/163 input

bin$ hadoop fs -put /home/dic/input/sina input

bin$ hadoop fs -ls input

3.运行wordcount

bin$ hadoop jar ../hadoop-0.20.2/hadoop-0.20.2-examples.jar wordcount input output

4.检测输出

bin$ hadoop fs -ls output

--------------------------------------------------------------------------------------------------------------

遇到的问题

1：运行wordcount程序，控制台没任何显示

原因：因为我的hadoop启动後默认进入safemode模式，该模式下不允许对文件进行修改和删除。所以使用

hadoop dfsadmin -safemode leave

离开安全模式

2：控制台报错

Exception in thread "main" java.io.IOException: Error opening job jar: hadoop-0.20.2-examples.jar
    at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
Caused by: java.util.zip.ZipException: error in opening zip file
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:131)
    at java.util.jar.JarFile.<init>(JarFile.java:150)
    at java.util.jar.JarFile.<init>(JarFile.java:87)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:88)

这是因为找不到 job jar 而出现的错误

正确命令

bin$ hadoop jar ../hadoop-0.20.2/hadoop-0.20.2-examples.jar wordcount input output

或者

$ bin/hadoop jar hadoop-0.20.1-examples.jar wordcount input output

3：控制台报错

org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory output already exists

这是因为mapreduce的输出都应该是单独的输出文件，不能有重名的情况