minFunc 2012:Huber loss

minFunc 2012:Huber loss

本Markdown编辑器使用[StackEdit][6]修改而来，用它写博客，将会带来全新的体验哦：

• Huber loss 定义

Huber loss 定义

The Huber loss function describes the penalty incurred by an estimation procedure f. Huber (1964) defines the loss function piecewise by[1]

Lδ(a)={−xδ（|a|−12δ）for|a|≤δ,otherwise.

This function is quadratic for small values of a, and linear for large values, with equal values and slopes of the different sections at the two points where |a|=δ . The variable a often refers to the residuals, that is to the difference between the observed and predicted values a=y−f(x) , so the former can be expanded to

Lδ(y,f(x))={−12(y−f(x))2δ（|y−f(x)|−12δ2）for|y−f(x)|≤δ,otherwise.

Huber loss (green, \delta=1) and squared error loss (blue) as a function of y - f(x)

<!--matlab-->
function [f,g,H,T] = HuberLoss(w,X,y,k)
% w(feature,1)
% X(instance,feature)
% y(instance,1)

r = X*w-y;

closeInd = abs(r) <= k;

f = (1/2)*sum(r(closeInd).^2) + k*sum(abs(r(~closeInd))) - (1/2)*sum(~closeInd)*k^2;
if nargout > 1
   g = X(closeInd,:)'*(X(closeInd,:)*w - y(closeInd)) + k*X(~closeInd,:)'*sign(r(~closeInd));
end

if nargout > 2
   H = X(closeInd,:)'*X(closeInd,:); 
end