实验四 SVM实验（模式识别与机器学习）
目录
实验1：线性SVM分类实验
实验2：具有高斯核的SVM
实验3：交叉验证
实验1：线性SVM分类实验
实验步骤：载入数据、可视化数据、完成线性SVM的训练，画出决策边界；改变C的值，观察决策边界如何变化
要求：实现linearKernel.m；实现visualizeBoundaryLinear.m 函数中的部分代码
linearKernel.m
function sim = linearKernel(x1, x2)
%LINEARKERNEL returns a linear kernel between x1 and x2
% sim = linearKernel(x1, x2) returns a linear kernel between x1 and x2
% and returns the value in sim
% Ensure that x1 and x2 are column vectors
x1 = x1(:); x2 = x2(:);
% Compute the kernel
%%-----------填空: 计算线性核----------------------------------------------
sim = x1' * x2; %点乘
endvisualizeBoundaryLinear.m
function visualizeBoundaryLinear(X, y, model)
%VISUALIZEBOUNDARYLINEAR plots a linear decision boundary learned by the
%SVM
% VISUALIZEBOUNDARYLINEAR(X, y, model) plots a linear decision boundary
% learned by the SVM and overlays the data on it
%可视化边界线性绘制SVM学习的线性决策边界
% 可视化边界线性（X， y， 模型） 绘制线性决策边界
% 由 SVM 学习并在其上叠加数据
w = model.w;
b = model.b;
xp = linspace(min(X(:,1)), max(X(:,1)), 100); %创建一个由区间 [min,max] 中的 100 个等距点组成的向量。
%%-------填空：根据模型计算yp----------------------------------------------
yp = -(w(1)*xp + b)/w(2); % w(1)xp+w(2)x+b=0
plotData(X, y);
hold on;
plot(xp, yp, '-b');
hold off
end
Exp_1
%% Initialization
clear ; close all; clc
%% =============== Part 1: Loading and Visualizing Data ================
% We start the exercise by first loading and visualizing the dataset.
% The following code will load the dataset into your environment and plot
% the data.
%
fprintf('Loading and Visualizing Data ...\n')
% Load from ex6data1:
% You will have X, y in your environment
load('ex6data1.mat');
% Plot training data
plotData(X, y);
fprintf('Program paused. Press enter to continue.\n');
pause;
% ==================== Part 2: Training Linear SVM ====================
% The following code will train a linear SVM on the dataset and plot the
% decision boundary learned.
% Load from ex6data1:
% You will have X, y in your environment
load('ex6data1.mat');
fprintf('\nTraining Linear SVM ...\n')
% You should try to change the C value below and see how the decision
% boundary varies (e.sg., try C = 1000)
C = 1;
model = svmTrain(X, y, C, @linearKernel, 1e-3, 20);
visualizeBoundaryLinear(X, y, model);
fprintf('Program paused. Press enter to continue.\n');
pause;实验2：具有高斯核的SVM
实验步骤：载入数据、可视化数据、实现高斯核函数完成SVM的训练、画出决策边界。
要求：实现gaussianKernel.m 函数
gaussianKernel.m
function sim = gaussianKernel(x1, x2, sigma)
%RBFKERNEL returns a radial basis function kernel between x1 and x2
% sim = gaussianKernel(x1, x2) returns a gaussian kernel between x1 and x2
% and returns the value in sim
% Ensure that x1 and x2 are column vectors
x1 = x1(:); x2 = x2(:);
% You need to return the following variables correctly.
sim = 0;
% ====================== YOUR CODE HERE ======================
% Instructions: Fill in this function to return the similarity between x1
% and x2 computed using a Gaussian kernel with bandwidth
% sigma
%
%
%%-----------填空: 计算高斯核----------------------------------------------
sim = exp(sum((x1 - x2) .^ 2) / 2 / sigma / sigma * (-1));
% =============================================================
end
Exp_2
%% =============== Part 1: Implementing Gaussian Kernel ===============
% You will now implement the Gaussian kernel to use
% with the SVM. You should complete the code in gaussianKernel.m
%
fprintf('\nEvaluating the Gaussian Kernel ...\n')
x1 = [1 2 1]; x2 = [0 4 -1]; sigma = 2;
sim = gaussianKernel(x1, x2, sigma);
fprintf(['Gaussian Kernel between x1 = [1; 2; 1], x2 = [0; 4; -1], sigma = 0.5 :' ...
'\n\t%f\n(this value should be about 0.324652)\n'], sim);
fprintf('Program paused. Press enter to continue.\n');
pause;
%% =============== Part 2: Visualizing Dataset 2 ================
% The following code will load the next dataset into your environment and
% plot the data.
%
fprintf('Loading and Visualizing Data ...\n')
% Load from ex6data2:
% You will have X, y in your environment
load('ex6data2.mat');
% Plot training data
plotData(X, y);
fprintf('Program paused. Press enter to continue.\n');
pause;
%% ========== Part 3: Training SVM with RBF Kernel (Dataset 2) ==========
% After you have implemented the kernel, we can now use it to train the
% SVM classifier.
%
fprintf('\nTraining SVM with RBF Kernel (this may take 1 to 2 minutes) ...\n');
% Load from ex6data2:
% You will have X, y in your environment
load('ex6data2.mat');
% SVM Parameters
C = 1000; sigma = 0.1;
% We set the tolerance and max_passes lower here so that the code will run
% faster. However, in practice, you will want to run the training to
% convergence.
model= svmTrain(X, y, C, @(x1, x2) gaussianKernel(x1, x2, sigma));
visualizeBoundary(X, y, model);
fprintf('Program paused. Press enter to continue.\n');
pause;
实验3：交叉验证
实验步骤：使用交叉验证寻找最优的学习参数 C 和sigma，完成SVM的训练，
画出决策边界。
要求：实现 cross_validation.m 函数部分代码
cross_validation.m
function [C, sigma] = cross_validation(X, y, Xval, yval, param)
%cross_validation returns your choice of C and sigma for Part 3 of the exercise
%where you select the optimal (C, sigma) learning parameters to use for SVM
%with RBF kernel
% [C, sigma] = cross_validation(X, y, Xval, yval) returns your choice of C and
% sigma. You should complete this function to return the optimal C and
% sigma based on a cross-validation set.
%
% ====================== YOUR CODE HERE ======================
% Instructions: Fill in this function to return the optimal C and sigma
% learning parameters found using the cross validation set.
% You can use svmPredict to predict the labels on the cross
% validation set. For example,
% predictions = ...;
% will return the predictions on the cross validation set.
%
% Note: You can compute the prediction error using
%
%
minError = 10000.0;
for CVal = param,
for sigmaVal = param,
%%------填空：计算预测标签与真实标签之间的误差（分类错误率）---------------------
model = svmTrain(X, y, CVal, @(x1, x2) gaussianKernel(x1, x2,sigmaVal));
predictions = svmPredict(model, Xval);
%sum(double(predictions ~= yval))
error =mean(double(predictions ~= yval));
if minError > error,
minError = error;
C = CVal;
sigma = sigmaVal;
end
end
end
% =========================================================================
end
Exp_3
%% =============== Part 1: Visualizing Dataset 3 ================
% The following code will load the next dataset into your environment and
% plot the data.
fprintf('Loading and Visualizing Data ...\n')
% Load from ex6data3:
% You will have X, y in your environment
load('ex6data3.mat');
% Plot training data
plotData(X, y);
fprintf('Program paused. Press enter to continue.\n');
pause;
%% ========== Part 2: Training SVM with RBF Kernel (Dataset 3) ==========
% This is a different dataset that you can use to experiment with. Try
% different values of C and sigma here.
%
% Load from ex6data3:
% You will have X, y in your environment
load('ex6data3.mat');
% Try different SVM Parameters here
% Exercise cross-validation
% Parameter set for C and sigma
param = [0.01 , 0.03, 0.1, 0.3, 1, 3, 10, 30];
[C, sigma] = cross_validation(X, y, Xval, yval, param); %X, y：training set; Xval，yval： Validation set
C , sigma
% Train the SVM
model= svmTrain(X, y, C, @(x1, x2) gaussianKernel(x1, x2, sigma));
visualizeBoundary(X, y, model);
fprintf('Program paused. Press enter to continue.\n');
pause;