Kintinuous + Elastic Fusion 配置运行方法
系统：Ubuntu 14
Kintinuous + Elastic Fusion代码来源：https://github.com/mp3guy
准备工作
安装git
sudo apt-get install git
git config --global user.name 你的git username
git config --global user.email 你的邮箱安装cmake
直接在ubuntu软件中心下载安装，但是配置Elastic Fusion需要升级到3.2，否则自带的是2.8
参考：http://askubuntu.com/questions/610291/how-to-install-cmake-3-2-on-ubuntu-14-04
sudo apt-get install build-essential
wget http://www.cmake.org/files/v3.2/cmake-3.2.2.tar.gz
tar xf cmake-3.2.2.tar.gz
cd cmake-3.2.2
./configure
make
sudo make install如果提示找不到bin/cmake，那么sudo apt-get upgrade就可以了
配置Kinect运行环境（一个参考，实际上我并没有用Kinect跑，最后用的是Xtion）
参考：http://www.cc.gatech.edu/grads/a/ahuaman3/docs/tutorials/software/build/perception.html
OpenNI2安装：
sudo apt-get install g++
sudo apt-get install python
sudo apt-get install libusb-1.0-0-dev
sudo apt-get install libudev-dev
sudo apt-get install openjdk-6-jdk
sudo apt-get install freeglut3-dev
sudo apt-get install graphviz
sudo apt-get install doxygen
git clone https://github.com/occipital/OpenNI2
cd OpenNI2
make在其中加入：
UBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02b0", MODE:="0666", OWNER:="root", GROUP:="video"
SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02ad", MODE:="0666", OWNER:="root", GROUP:="video"
SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02ae", MODE:="0666", OWNER:="root", GROUP:="video"
SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02c2", MODE:="0666", OWNER:="root", GROUP:="video"
SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02be", MODE:="0666", OWNER:="root", GROUP:="video"
SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02bf", MODE:="0666", OWNER:="root", GROUP:="video"libfreenect安装：
git clone https://github.com/OpenKinect/libfreenect.git
cd libfreenect
mkdir build; cd build
cmake .. -DBUILD_OPENNI2_DRIVER=ON
make
cp -L lib/OpenNI2-FreenectDriver/libFreenectDriver.so ${OPENNI2_DIR}/Bin/x64-Release/OpenNI2/Drivers #将驱动拷贝到OpenNI2的解压文件夹下，OPENNI2_DIR：OpenNI2的解压文件夹插上kinect后，使用lsusb命令查看确保有以下三个：
Xbot camera
Xbot motor
Xbot audio
安装成功检查：
cd ${OPENNI2_DIR}/Bin/x64-Release/
./NiViewer配置Kintinuours运行环境
git clone https://github.com/mp3guy/Kintinuous1.新系统：
找到build.sh，在属性中设置为可执行
./build.sh
等待配置成功
2.非新系统：
安装CUDA：
下载deb文件 https://developer.nvidia.com/cuda-downloads
sudo dpkg -i cuda-repo-ubuntu1404_7.5-18_amd64.deb
sudo apt-get update
sudo apt-get install cuda
sudo rebootcd /usr/local/cuda/samples
sudo make all -j4
cd /usr/local/cuda/samples/bin/x86_64/linux/release
sudo ./deviceQueryCUDA Device Query (Runtime API) version (CUDART static linking)
Detected 1 CUDA Capable device(s)
我在这里遇到了一定问题，找了很久，发现是Secure Boot没关掉，关掉之后就好了。解决的流程如下：
运行deviceQuery：no capable device
尝试解决：
1.怀疑是内核问题
add in: GRUB_CMDLINE_LINUX_DEFAULT with: pci=nocrs pci=realloc
依然：no capable device
2.装最新的nvidia 367
新问题：CUDA driver version is insufficient for CUDA runtime version
3.查看当前的kernel版本
kernel:3.19.0-65-generic
4.查看一些参数
sudo dpkg --list | grep nvidia
lspci | grep -i nvidia 设备里的确有nvidia显卡
lsmod 发现video一览只有i915_bpo，说明nvidia独显并没有被启用
sudo modprobe nvidia-367 手动启用nvidia显卡，提示ERROR: could not insert 'nvidia_367': Required key not available
终于找到问题：不是显卡驱动没装好，而是无法启动显卡（被BIOS禁了？）
5.装回nvidia 352
sudo apt-get remove --purge nvidia-*
(***提示cmake-qt-gui依赖cmake 2.8出错，卸载cmake-qt-gui。卸载后cmake命令没出错，其他情况会不会出错有待观察)
sudo add-apt-repository ppa:xorg-edgers/ppa -y
sudo apt-get update
sudo apt-get install nvidia-352
6.查看安装的版本
sudo dpkg --list | grep nvidia
ii nvidia-352 352.93-0ubuntu1 amd64 NVIDIA binary driver - version 352.93
ii nvidia-modprobe 352.93-0ubuntu1 amd64 Load the NVIDIA kernel driver and create device files
ii nvidia-opencl-icd-352 352.93-0ubuntu1 amd64 NVIDIA OpenCL ICD
ii nvidia-prime 0.6.2 amd64 Tools to enable NVIDIA's Prime
ii nvidia-settings 367.35-0ubuntu0~gpu14.04.1 amd64 Tool for configuring the NVIDIA graphics driver
安装正确，运行devieQuery依然提示no device found，回到原点
7.现在的问题：
sudo modprobe nvidia-352 手动启用nvidia显卡，提示ERROR: could not insert 'nvidia_352': Required key not available
网上给出的解决方法：turn off the secure boot feature of the motherboard
8.尝试解决：开机F12，secure boot勾选为disable，legacy勾选
sudo modprobe nvidia-352 成功！
nvidia-settings 显示独显成功！
deviceQuery 检测到显卡！应该可以了！
9.新问题：kinect无法连接了。。。（之前是可以的）
lsusb 能看到XBOX的三个设备
从头重新配置，发现有几个依赖包没有安装（可能之前哪一步把他卸载了）
sudo apt-get install openjdk-6-jdk
sudo apt-get install graphviz
sudo apt-get install doxygen
之后NiViewer可以正常运行了
安装依赖项：
sudo apt-get install -y cmake-qt-gui git build-essential libusb-1.0-0-dev libudev-dev openjdk-7-jdk freeglut3-dev python-vtk libvtk-java libglew-dev cuda-7-5 libsuitesparse-dev安装pcl：
sudo add-apt-repository ppa:v-launchpad-jochen-sprickerhof-de/pcl
sudo apt-get update
sudo apt-get install libpcl-all安装opengl：
sudo apt-get install build-essential
sudo apt-get install libgl1-mesa-dev
sudo apt-get install libgl1-mesa-dev
sudo apt-get install libglu1-mesa-dev
sudo apt-get install freeglut3-dev安装opencv：
（不可以直接在ubuntu的apt-get里面获取opencv，因为ubuntu提供的opencv没有nonfree模块）
下载http://opencv.org/downloads.html
sudo apt-get install build-essential libgtk2.0-dev libjpeg-dev libtiff4-dev libjasper-dev libopenexr-dev cmake python-dev python-numpy python-tk libtbb-dev libeigen2-dev yasm libfaac-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev libx264-dev libqt4-dev libqt4-opengl-dev sphinx-common texlive-latex-extra libv4l-dev libdc1394-22-dev libavcodec-dev libavformat-dev libswscale-dev
unzip opencv-2.4.9.zip
cd opencv-2.4.9
mkdir build
cd build
cmake -D BUILD_NEW_PYTHON_SUPPORT=OFF -D WITH_OPENCL=OFF -D WITH_OPENMP=ON -D INSTALL_C_EXAMPLES=OFF -D BUILD_DOCS=OFF -D BUILD_EXAMPLES=OFF -D WITH_QT=OFF -D WITH_OPENGL=OFF -D WITH_VTK=OFF -D BUILD_PERF_TESTS=OFF -D BUILD_TESTS=OFF -D WITH_CUDA=OFF -D BUILD_opencv_gpu=OFF ..
make
sudo make install安装Dlib：
sudo apt-get install libboost-dev
git clone https://github.com/dorian3d/DLib
cd DLib/build
cmake ..
make
sudo make install安装DBoW2：
git clone https://github.com/dorian3d/DBoW2#dbow2
cd DBoW2/build
cmake ..
make
sudo make install安装DLoopDetector：
git clone https://github.com/dorian3d/DLoopDetector
cd DLoopDetector/build
cmake ..
make
sudo make install安装iSAM：
svn co https://svn.csail.mit.edu/isam
cd isam
make安装Pangolin：
sudo apt-get install libglew-dev
sudo apt-get install libboost-dev libboost-thread-dev libboost-filesystem-dev
sudo apt-get install libpython2.7-dev
sudo apt-get install ffmpeg libavcodec-dev libavutil-dev libavformat-dev libswscale-dev
sudo apt-get install libdc1394-22-dev libraw1394-dev
sudo apt-get install libjpeg-dev libpng12-dev libtiff5-dev libopenexr-dev
git clone https://github.com/stevenlovegrove/Pangolin
cd Pangolin/build
cmake -DCPP11_NO_BOOST=1 ..
make -j运行Kintinuous：
作者的一些说明：
The GUI (Kintinuous) can take a bunch of parameters when launching it from the command line. They are as follows:
-c : Loads a camera calibration file specified by a depth_intrinsics matrix in OpenCV format.
-l : Processes the specified .klg log file.
-v : Loads DBoW vocabulary file.
-p : Loads ground truth poses to use instead of estimated pose.
-gpu : Sets which GPU should be used by CUDA.
-n : Number of frames to process.
-t : Voxel threshold for volume shifting (default 14).
-cw : Removes voxels below this threshold when extracting slices (default 8).
-lt : Disallow loop closures within this time period of the last (default 30s).
-s : Size of the fusion volume (default 6m).
-dg : Rate of pose sampling for deformation (default 0.8m).
-il : Inlier threshold for RANSAC (default 0.35).
-it : Residual threshold for pose graph optimisation (default 10).
-sm : Static mode (disable volume shifting).
-f : Flip RGB/BGR.
-od : Perform online deformation (required for loop closure).
-m : Enable mesh generation.
-no : Disable overlap of extracted slices.
-nos : Remove overlap when saving map.
-r : Use RGB tracking only.
-ri : Use combined ICP+RGB tracking.
-d : Enable dynamic cube positioning.
-dc : Disable color weighting by angle.
-fl : Subsample pose graph for faster loop closure.
-fod : Enable fast odometry.
以上所有配置以后就可以正常运行kintinuous了，运行的命令可以看上面的作者说明，比如：
./Kintinuous 默认从华硕Xtion中读取图像，如果没有Xtion则会报错找不到设备
没有话说Xtion的可以试试作者给的几个log，在原网址上有下载。
使用log的运行方法如下：
./Kintinuous -s 7 -v ../vocab.yml.gz -l loop.klg -ri -fl -od # author has provided a sample dataset loop.klg which you can run easily with Kintinuous
配置ElasticFusion
准备工作和相关的依赖项包括开源包的配置方法和KIntinous一样，只需要额外安装开源包openni2
安装OpenNI2:git clone https://github.com/occipital/OpenNI2.git
cd OpenNI2
make -j8
cd ..
生成 ./ElasticFusion
注意源码里有三个项目，Core、GPUTest和GUI
如果要使用作者的程序，则需要把三个项目安装顺序依次配置成功
cd ../Core
mkdir build
cd build
cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-7.5 ../src
make -j8
cd ../../GPUTest
mkdir build
cd build
cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-7.5 ../src
make -j8
cd ../../GUI
mkdir build
cd build
cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-7.5 ../src
make -j8
cd ../../myfusion
mkdir build
cd build
cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-7.5 ../src
make -j8直接调用以下命令，默认从华硕Xtion中获取图像并开始三位重建
./ElasticFusion也可以使用参数控制，一些Elastci Fusion的运行参数如下：
- -cal : Loads a camera calibration file specified as fx fy cx cy. 载入相机参数（默认参数为华硕Xtion的参数）
- -l : Processes the specified .klg log file. 不使用摄像头读取，使用log文件模拟相机的图像采集过程
- -p : Loads ground truth poses to use instead of estimated pose. 读取ground truth的位姿来作为三位重建的依据之一（默认会自动估计当前的位置，如果传入ground truth的位姿则直接使用传入的位姿）
- -c : Surfel confidence threshold (default 10). Surfel的信任阈值
- -d : Cutoff distance for depth processing (default 3m). 多少距离以上的深度数据视为无用数据（单位mm，默认为3000，即3m）
- -i : Relative ICP/RGB tracking weight (default 10). ICP/RGB的tracking阈值
- -ie : Local loop closure residual threshold (default 5e-05).
- -ic : Local loop closure inlier threshold (default 35000).
- -cv : Local loop closure covariance threshold (default 1e-05).
- -pt : Global loop closure photometric threshold (default 115).
- -ft : Fern encoding threshold (default 0.3095). 关键帧编码阈值
- -t : Time window length (default 200).
- -s : Frames to skip at start of log. 如果使用log，跳过初始帧的数目
- -e : Cut off frame of log.
- -f : Flip RGB/BGR.
- -icl : Enable this if using the ICL-NUIM dataset (flips normals to account for negative focal length on that data).
- -o : Open loop mode.
- -rl : Enable relocalisation.
- -fs : Frame skip if processing a log to sim
- ulate real-time.
- -q : Quit when finished a log.
- -fo : Fast odometry (single level pyramid).
- -nso : Disables SO(3) pre-alignment in tracking.
- -r : Rewind and loop log forever.
- -ftf : Do frame-to-frame RGB tracking.
- -sc : Showcase mode (minimal GUI).
使用Elastic Fusion的CoreAPI
作者还提供了使用他的Core作为基础库的方法。
首先编译Core项目
cd ../Core
mkdir build
cd build
cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-7.5 ../src
make -j8然后在CMakelist.txt中加入
find_package(efusion REQUIRED)
include_directories(${EFUSION_INCLUDE_DIR})
target_link_libraries(MyProject ${EFUSION_LIBRARY})在头文件中加入
#include 初始化
Resolution::getInstance(640, 480); #图像大小
Intrinsics::getInstance(528, 528, 320, 240); #相机参数
pangolin::Params windowParams; #GUI窗口
windowParams.Set("SAMPLE_BUFFERS", 0);
windowParams.Set("SAMPLES", 0);
pangolin::CreateWindowAndBind("Main", width, height, windowParams);
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
glPixelStorei(GL_PACK_ALIGNMENT, 1);使用封装好的ElasticFusion类，可以精确定义每一个参数
默认的构造函数
ElasticFusion eFusion; 精确定义每一个参数后调用构建函数
float confidence = 10.0f; //fusion的confidence阈值
float depth = 3.0f; //去掉depth大于某个阈值的帧
float icp = 10.0f; //icp的阈值
float icpErrThresh = 5e-05; //icp错误阈值
float covThresh = 1e-05;
float photoThresh = 115;
float fernThresh = 0.3095f; //新fern的阈值
int timeDelta = 200;
int icpCountThresh = 35000;
bool openLoop = 0; //open loop模式：不开启
bool iclnuim = 0; //使用icl dataset:不使用
bool reloc = 0; //重定位模式：不开启
bool fastOdom = 0; //Fast odometry (single level pyramid) mode :不开启
bool so3 = 1; //SO(3) pre-alignment in tracking：开启
bool frameToFrameRGB = 0; //只做rgb图像的tracking：不开启
int timestamp = 0;
std::string savefilename = "test";
ElasticFusion eFusion(openLoop ? std::numeric_limits::max() / 2 : timeDelta,icpCountThresh, icpErrThresh, covThresh, !openLoop, iclnuim, reloc,photoThresh, confidence, depth, icp, fastOdom, fernThresh, so3,frameToFrameRGB, savefilename); 处理输入帧：
eFusion.processFrame(rgb, depth, timestamp, currentPose, weightMultiplier);可选参数1:currentPose——先验的current pose,比如传入了标定好的ground truth三维坐标数据,则可以用上一时刻和这一时刻的三位坐标来估计相机姿态
Eigen::Matrix4f * currentPose;//当前的位姿
currentPose = new Eigen::Matrix4f;
currentPose->setIdentity();
*currentPose = groundTruthOdometry->getTransformation(timestamp); }
eFusion.processFrame(rgb, depth, timestamp, currentPose, weightMultiplier);
delete currentPose; 可选参数2:weightMultiplier (full frame fusion weight)——默认为1.f 可以自己调整,比如针对前面跳过的帧数对参数进行调整
float weightMultiplier = framesToSkip + 1;
eFusion.processFrame(rgb, depth, timestamp, currentPose, weightMultiplier);一个简单的使用CoreAPI的程序：
#include "myfusion_base.h"
int main() {
//图像的尺寸以及相机参数
int width = 640;
int height = 480;
int camera_fx = 528;
int camera_fy = 528;
int camera_cx = 320;
int camera_cy = 240;
cout << "Initializing ..." << endl;
Resolution::getInstance(width, height);
Intrinsics::getInstance(camera_fx, camera_fy, camera_cx, camera_cy);
cout << "Initializing done." << endl;
cout << "Setting parameters..." << endl;
float confidence = 10.0f; //fusion的confidence阈值
float depth = 3.0f; //去掉depth大于某个阈值的帧
float icp = 10.0f; //icp的阈值
float icpErrThresh = 5e-05; //icp错误阈值
float covThresh = 1e-05;
float photoThresh = 115;
float fernThresh = 0.3095f; //新fern的阈值
int timeDelta = 200;
int icpCountThresh = 35000;
//int start = 1;
//int end = std::numeric_limits::max(); //Funny bound, since we predict times in this format really!
bool openLoop = 0; //open loop模式：不开启
bool iclnuim = 0; //使用icl dataset:不使用
bool reloc = 0; //重定位模式：先做重建，不开启重定位
bool fastOdom = 0; //Fast odometry (single level pyramid) mode :不开启
bool so3 = 1; //SO(3) pre-alignment in tracking：开启
bool frameToFrameRGB = 0; //只做rgb图像的tracking：不开启
int timestamp = 0;
std::string savefilename = "test";
cout << "Setting parameters done." << endl;
cout << "Building eFusion..." << endl;
pangolin::Params windowParams;
windowParams.Set("SAMPLE_BUFFERS", 0);
windowParams.Set("SAMPLES", 0);
pangolin::CreateWindowAndBind("Main", width, height, windowParams);
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
glPixelStorei(GL_PACK_ALIGNMENT, 1);
ElasticFusion eFusion(
openLoop ? std::numeric_limits::max() / 2 : timeDelta,
icpCountThresh, icpErrThresh, covThresh, !openLoop, iclnuim, reloc,
photoThresh, confidence, depth, icp, fastOdom, fernThresh, so3,
frameToFrameRGB, savefilename);
cout << "Building eFusion done" << endl;
//待处理文件的位置和下标
std::string filedir = "../pic/";
int file_start = 1;
int file_end = 782;
vector pose;
//三位重建
for (int file_ptr = file_start; file_ptr <= file_end; file_ptr = file_ptr +1) {
cout << "Processing frame : "< eFusion.confidenceThreshold) {
//这是个有效的顶点,validCount++;
}
}//调用点云中的每一个点
*/
cout << "saving cloud points..." << endl;
eFusion.savePly(); //保存当前的点云图至ply
cout << "cloud point has saved to " << savefilename << ".ply" << endl;
return 0;
}
FRAME readFrame(int index, std::string filedir) {
FRAME f;
// 文件目录
string rgbDir = filedir + "rgb/";
string depthDir = filedir + "depth/";
// 文件后缀
string rgbExt = ".png";
string depthExt = ".png";
stringstream ss;
// 读rgb图
ss << rgbDir << index << rgbExt;
string filename;
ss >> filename;
f.rgb_mat = cv::imread(filename);
f.rgb = f.rgb_mat.data;
// 读depth图
ss.clear();
filename.clear();
ss << depthDir << index << depthExt;
ss >> filename;
f.dep_mat = cv::imread(filename, -1);
f.dep = (unsigned short*) f.dep_mat.data;
f.frameID = index;
return f;
}