Hive insert overwrite时设置lzo压缩格式报错：NATIVE-LZO LIBRARY NOT AVAILABLE 解决方案

最近在将hive中的临时表数据insert overwrite到ods表中去的时候，设置为lzo的压缩格式，报错如下：

Caused by: java.lang.RuntimeException: native-lzo library not available

针对"native-lzo library not available" 异常即lzo安装的异常。

查了很多帖子，最终整合了解决方法。

1.将 HADOOP-LZO-0.4.20放到 每个机器的 /HADOOP-2.7.2/SHARE/HADOOP/COMMON/ 目录下

将编译后得到的hadoop-lzo-0.4.20.jar放到我的三台机器安装了hadoop的share/hadoop/common目录下。

2.修改各个HADOOP节点的HADOOP的配置文件:

cd /opt/module/hadoop-2.7.2/etc/hadoop
vi hadoop-env.sh 

# Extra Java CLASSPATH elements.  Optional.
export HADOOP_CLASSPATH=":$HADOOP_CLASSPATH:${HADOOP_HOME}/share/hadoop/common"
#export JAVA_LIBRARY_PATH=${JAVA_LIBRARY_PATH}:/opt/module/hadoop-2.7.2/lib/native:/opt/glibc-2.14/lib
export JAVA_LIBRARY_PATH=${JAVA_LIBRARY_PATH}:/opt/module/hadoop-2.7.2/lib/native

3.在 CORE-SITE.XML <-CONFIGURATION-> 添加以下配置

vi core-site.xml

<property>
        <name>io.compression.codecsname>
        <value>
        org.apache.hadoop.io.compress.GzipCodec,
        org.apache.hadoop.io.compress.DefaultCodec,
        org.apache.hadoop.io.compress.BZip2Codec,
        org.apache.hadoop.io.compress.SnappyCodec,
        com.hadoop.compression.lzo.LzoCodec,
        com.hadoop.compression.lzo.LzopCodec
        value>
property>

<property>
        <name>io.compression.codec.lzo.classname>
        <value>com.hadoop.compression.lzo.LzoCodecvalue>
property>

4. 在 MAPRED-SITE.XML <-CONFIGURATION-> 添加以下配置

vi mapred-site.xml

<property>
<name>mapred.compress.map.outputname>
<value>truevalue>
property>

<property>
<name>mapred.map.output.compression.codecname>
<value>com.hadoop.compression.lzo.LzoCodecvalue>
property>

<property>
<name>mapreduce.map.envname>
<value>LD_LIBRARY_PATH=/opt/module/hadoop-2.7.2/lzo/libvalue>
property>

<property>
<name >mapreduce.reduce.envname>
<value>LD_LIBRARY_PATH=/opt/module/hadoop-2.7.2/lzo/libvalue>
property>

<property>
<name>mapred.child.envname>
<value>LD_LIBRARY_PATH=/opt/module/hadoop-2.7.2/lzo/libvalue>
property>

然后重启hdfs和yarn，再执行insert overwrite，数据导入成功。