logistical regression

1.logistic regression model

1.1 classification

want

1.2 cost function

y=1

y=0

simplified cost function:

gradient descent:
repeat {

}

1.3Optimization algorithms:

• Gradient descent
• Conjugate gradient
• BFGS
• L-BFGS
  cost function

function [jVal, gradient] = costFunction(theta)
  jVal = [...code to compute J(theta)...];
  gradient = [...code to compute derivative of J(theta)...];
end

Then we can use octave's "fminunc()" optimization algorithm along with the "optimset()" function that creates an object containing the options we want to send to "fminunc()"

options = optimset('GradObj', 'on', 'MaxIter', 100);
initialTheta = zeros(2,1);
   [optTheta, functionVal, exitFlag] = fminunc(@costFunction, initialTheta, options);

1.4 multiclass classfication

Train a logistic regression classifier for each class to predict the probability that y = i .To make a prediction on a new x, pick the class that maximizes

1.5 how to solve overFitting

• reduce number of features
• regularization

1.5.1 Regularized Linear Regression

small values for parameters
①simpler hypothesis
②less prone to overfitting

if is too large,then it will result in underfitting, because will be close to 0 at this moment.

gradient descent:
repeat{


}

Normal Equation:

normal equation

1.5.2 Regularized logistic Regression

attention:

gradient descent:
repeat{


}

---

Appendices

the derivation of cost function
first：


so we can get the general formula:

then use the Maximum likelihood estimation(MLE):
note that
substitute equation (1) into equation (2):

on equation (3) on both sides of the natural logarithm:

we know that MLE's goal is to get the best that makes equation (4) max, so we let

next, we will to get the deviation:

so further,we can calculate :