启动bin/spark-shell警告:WARN NativeCodeLoader: Unable to load native-hadoop library for your platform.

启动bin/spark-shell警告:WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

原因:spark告诉你系统无法加载本地的Hadoop库,但仍然会使用内置的Java类来代替。

解决:在spark目录conf/spark-env.sh文件,末尾添加如下内容:

export LD_LIBRARY_PATH=$HADOOP_HOME/lib/native

你可能感兴趣的:(spark,hadoop,大数据)