神经网络实现手写字体识别

神经网络入门学习中，进行了手写字体识别实践，该篇博客用于记录实践代码，以备后续使用。

关键词：神经网络，前向传播、反向传播、梯度下降、权值更新、手写字体识别

1. 实践代码

import numpy as np
from sklearn.datasets import load_digits
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
import matplotlib.pyplot as plt

# 载入数据
digits = load_digits()
# 显示图片
for i in range(min(digits.images.shape[0], 2)):
    plt.imshow(digits.images[i], cmap='gray')
    plt.show()

# 数据
X = digits.data
# 标签
y = digits.target

# 定义一个神经网络，结构，64-100-
# 定义输入层到隐藏层之间的权值矩阵
V = np.random.random((64, 100)) * 2 - 1
# 定义隐藏层到输出层之间的权值矩阵
W = np.random.random((100, 10)) * 2 - 1

# 数据切分
# 1/4为测试集，3/4为训练集
X_train, X_test, y_train, y_test = train_test_split(X, y)

# 标签二值化
# 0 -> 1000000000
# 3 -> 0003000000
# 9 -> 0000000001
labels_train = LabelBinarizer().fit_transform(y_train)


# 激活函数
def sigmoid(x):
    return 1 / (1 + np.exp(-x))


# 激活函数的导数
def dsigmoid(x):
    return x * (1 - x)


#  训练模型
def train(X, y, steps=10000, lr=0.011):
    global V, W
    for n in range(steps + 1):
        # 随机选取一个数据
        i = np.random.randint(X.shape[0])
        # 获取一个数据
        x = X[i]
        x = np.atleast_2d(x)
        # BP算法公式
        # 计算隐藏层的输出
        L1 = sigmoid(np.dot(x, V))
        # 计算输出层的输出
        L2 = sigmoid(np.dot(L1, W))
        # 计算L2_delta,L1_delta
        L2_delta = (y[i] - L2) * dsigmoid(L2)
        L1_delta = L2_delta.dot(W.T) * dsigmoid(L1)
        # 更新权值
        W += lr * L1.T.dot(L2_delta)
        V += lr * x.T.dot(L1_delta)

        # 每训练1000次预测一次准确率
        if n % 1000 == 0:
            output = predict(X_test)
            predictions = np.argmax(output, axis=1)
            acc = np.mean(np.equal(predictions, y_test))
            dW = L1.T.dot(L2_delta)
            dV = x.T.dot(L1_delta)
            gradient = np.sum([np.sqrt(np.sum(np.square(j))) for j in [dW, dV]])
            print('steps', n, 'accuracy', acc, 'gradient', gradient)
            # print(classification_report(predictions,y_test))


def predict(x):
    # 计算隐藏层的输出
    L1 = sigmoid(np.dot(x, V))
    # 计算输出层的输出
    L2 = sigmoid(np.dot(L1, W))
    return L2


# 开始训练
train(X_train, labels_train, 20000, lr=0.11)
train(X_train, labels_train, 20000, lr=0.011)
# 训练后结果对比
output = predict(X_test)
predictions = np.argmax(output, axis=1)
acc = np.mean(np.equal(predictions, y_test))
print('accuracy', acc)
print(classification_report(predictions, y_test, digits=4))

2. 运行结果

steps 1000 accuracy 0.39555555555555555 gradient 1.1540293157343329
steps 2000 accuracy 0.5822222222222222 gradient 0.24060995286476866
steps 3000 accuracy 0.6822222222222222 gradient 0.5377184040200084
steps 4000 accuracy 0.7088888888888889 gradient 0.3105382024238891
steps 5000 accuracy 0.7422222222222222 gradient 0.6367500261759913
steps 6000 accuracy 0.8288888888888889 gradient 0.7925874111147599
steps 7000 accuracy 0.8311111111111111 gradient 0.4219865590227713
steps 8000 accuracy 0.8866666666666667 gradient 0.19398854322579862
steps 9000 accuracy 0.9088888888888889 gradient 0.059956981030795414
steps 10000 accuracy 0.94 gradient 0.060926927213079046
steps 11000 accuracy 0.9222222222222223 gradient 0.0940042676753109
steps 12000 accuracy 0.9488888888888889 gradient 0.007252581282821429
steps 13000 accuracy 0.9422222222222222 gradient 0.0005117412876631392
steps 14000 accuracy 0.9577777777777777 gradient 0.020721130471145828
steps 15000 accuracy 0.9177777777777778 gradient 0.16479326937586997
steps 16000 accuracy 0.9422222222222222 gradient 3.7564619456548263
steps 17000 accuracy 0.9644444444444444 gradient 0.06896903217933802
steps 18000 accuracy 0.9577777777777777 gradient 0.027949995263778806
steps 19000 accuracy 0.9688888888888889 gradient 0.0006993893497620089
steps 20000 accuracy 0.9622222222222222 gradient 0.20674307169673445
steps 0 accuracy 0.9622222222222222 gradient 0.00872206168693661
steps 1000 accuracy 0.9666666666666667 gradient 0.022782881906241878
steps 2000 accuracy 0.9666666666666667 gradient 0.014290994295273446
steps 3000 accuracy 0.9688888888888889 gradient 0.48748927217168325
steps 4000 accuracy 0.9666666666666667 gradient 0.005666292598739775
steps 5000 accuracy 0.9688888888888889 gradient 0.0443512717424438
steps 6000 accuracy 0.9688888888888889 gradient 0.0061561535761366
steps 7000 accuracy 0.9688888888888889 gradient 0.020829652956511197
steps 8000 accuracy 0.9688888888888889 gradient 0.19995065353583805
steps 9000 accuracy 0.9688888888888889 gradient 0.5650831905730958
steps 10000 accuracy 0.9688888888888889 gradient 0.003948413021913768
steps 11000 accuracy 0.9688888888888889 gradient 0.13861557486472081
steps 12000 accuracy 0.9688888888888889 gradient 0.010983126563059315
steps 13000 accuracy 0.9711111111111111 gradient 0.024410425339698696
steps 14000 accuracy 0.9711111111111111 gradient 0.001296347083650798
steps 15000 accuracy 0.9711111111111111 gradient 0.0018664511288084774
steps 16000 accuracy 0.9711111111111111 gradient 0.0032670292289213856
steps 17000 accuracy 0.9711111111111111 gradient 0.017069352701509097
steps 18000 accuracy 0.9688888888888889 gradient 0.0033675869477754346
steps 19000 accuracy 0.9688888888888889 gradient 0.00043959401991603385
steps 20000 accuracy 0.9688888888888889 gradient 0.09278100040345633
accuracy 0.9688888888888889
              precision    recall  f1-score   support

           0     1.0000    1.0000    1.0000        43
           1     0.9787    1.0000    0.9892        46
           2     1.0000    1.0000    1.0000        37
           3     0.9787    0.9388    0.9583        49
           4     0.9091    1.0000    0.9524        50
           5     0.9800    0.9800    0.9800        50
           6     1.0000    0.9375    0.9677        48
           7     0.9565    1.0000    0.9778        44
           8     0.9189    0.9714    0.9444        35
           9     0.9767    0.8750    0.9231        48

   micro avg     0.9689    0.9689    0.9689       450
   macro avg     0.9699    0.9703    0.9693       450
weighted avg     0.9701    0.9689    0.9687       450