No.7 Regularization

The Problem of Overfitting

---

Example 1:   Linear Regression(Housing)

Example 2:  Logistic Regression

（g：sigmoid function）

Addressing Overfitting:  lots of features and very little training data,then,overfitting can be a problem

The Solution:

Option 1: Reduce number of features

               (1)Manually select which features to keep

               (2)Model selection algorithm(模型选择算法)

Option 2:Regularization

                (1)Keep all features, but reduce magnitude/ values 

                (2)Works well when we have a lot features

Cost Function

---

Example：   Housing

——Features:  x1 ,x2, x3, x4, x5, ....(Such as: the number of rooms, age,size,length...)

——Parameters: theta 1, theta 2, theta 3, ....

Regularized Linear Regression

---

Way 1:  Gradient descent

Cost Function:

Target:

Repeat:{

}

Way 2: Normal equation

Advantage:

1.take care of any non-invertibility issues

2. to avoid overfitting eveb if you have a lot of features in a relatively small training set

Regularized Logistic Regression 

---

Repeat:

{

}

---