TensorFlow2.0之还原lenet-5模型
最近课程需要，将lenet-5模型进行还原，但是网上博客大都根据原论文进行了自己的改进，不能满足还原的这个要求，所以按照论文结合TensorFlow2.0的API，将其还原。
from tensorflow.python.client import device_lib
device_lib.list_local_devices()
The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.
We recommend you upgrade now or ensure your notebook will continue to use TensorFlow 1.x via the %tensorflow_version 1.x magic: more info.
[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 16313844761984461564, name: "/device:XLA_CPU:0"
device_type: "XLA_CPU"
memory_limit: 17179869184
locality {
}
incarnation: 17583002155353847962
physical_device_desc: "device: XLA_CPU device", name: "/device:XLA_GPU:0"
device_type: "XLA_GPU"
memory_limit: 17179869184
locality {
}
incarnation: 188667365564072257
physical_device_desc: "device: XLA_GPU device", name: "/device:GPU:0"
device_type: "GPU"
memory_limit: 15956161332
locality {
bus_id: 1
links {
}
}
incarnation: 540513892452511862
physical_device_desc: "device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0"]
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
%matplotlib inline
print(tf.__version__)
1.15.0
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()
# np.newaxis增加一维数据
X_val, y_val = X_train[55000:, ..., np.newaxis], y_train[55000:]
X_train, y_train = X_train[:55000, ..., np.newaxis], y_train[:55000]
X_test = X_test[..., np.newaxis]
print("图片形状：{}".format(X_train.shape), end = '\n\n')
print("训练集合：{}".format(len(X_train)))
print("验证集合：{}".format(len(X_val)))
print("测试集合：{}".format(len(X_test)))
图片形状：(55000, 28, 28, 1)
训练集合：55000
验证集合：5000
测试集合：10000
# 将图片大小reshape为32*32，第1维=数量前面扩充0列，最后面扩充0列；第2维==行，前面扩充2列，最后面扩充2列；依次第3维=列、第4维...constant_values=未指定，扩充全为0
X_train = np.pad(X_train, ((0,0),(2,2),(2,2),(0,0)), 'constant')
X_val = np.pad(X_val, ((0,0),(2,2),(2,2),(0,0)), 'constant')
X_test = np.pad(X_test, ((0,0),(2,2),(2,2),(0,0)), 'constant')
print("图片形状更改为：", end='\n\n')
print("-训练数据集合{}".format(X_train.shape))
print("-验证数据集合{}".format(X_val.shape))
print("-测试数据集合{}".format(X_test.shape))
图片形状更改为：
-训练数据集合(55000, 32, 32, 1)
-验证数据集合(5000, 32, 32, 1)
-测试数据集合(10000, 32, 32, 1)
# 归1化
X_train, X_val, X_test = X_train/float(255), X_val/float(255), X_test/float(255)
"""默认自然图像是一类平稳的数据分布(即数据每一维的统计都服从相同分布)，此时，在每个样本上减去数据的统计平均值可以移除共同的部分，凸显个体差异。"""
X_train -= np.mean(X_train)
X_val -= np.mean(X_val)
X_test -= np.mean(X_test)
def plot_example(X, y, y_pred=None):
"""
展示9个训练样本及其各自类别
Parameters:
-X: 训练样本
-y: 真实标签
-y_pred: 预测标签
"""
# 3*3的显示阵列
fig,axes = plt.subplots(3, 3)
fig.subplots_adjust(hspace=0.3, wspace=0.3)
X,y=X[:9, ..., 0], y[:9]
for i, ax in enumerate(axes.flat):
# Plot image.
ax.imshow(X[i])
# Show true and predicted classes.
if y_pred is None:
xlabel = "True: {0}".format(y[i])
else:
# 显示预测结果
xlabel = "True: {0}, Pred: {1}".format(y[i], y_pred[i])
# Show the classes as the label on the x-axis.
ax.set_xlabel(xlabel)
# Remove ticks from the plot.
ax.set_xticks([])
ax.set_yticks([])
# Ensure the plot is shown correctly with multiple plots in a single Notebook cell.
plt.show()
def plot_example_errors(X, y, y_pred):
"""
Plots 9 example errors and their associate true/predicted labels.
Parameters:
-X: Training examples.
-y: true labels.
-y_pred: predicted labels.
"""
incorrect = (y != y_pred)
X = X[incorrect]
y = y[incorrect]
y_pred = y_pred[incorrect]
# Plot the first 9 images.
plot_example(X, y, y_pred)
plot_example(X_train, y_train)
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Conv2D, AveragePooling2D, Flatten, Dense
from tensorflow.keras.losses import CategoricalCrossentropy
def LeNet_5():
model = Sequential()
# C1: (None,32,32,1) -> (None,28,28,6).
model.add(Conv2D(6, kernel_size=(5, 5), strides=(1, 1), activation='tanh', input_shape=(32,32,1), padding='valid'))
"""
padding = “SAME”, 输出大小==向上取整[输入大小/步长]
padding = “VALID”，输出大小==向上取整[(输入大小-滤波器+1)/步长]
"""
# P1: (None,28,28,6) -> (None,14,14,6).
model.add(AveragePooling2D(pool_size=(2, 2), strides=(2, 2), padding='valid'))
# C2: (None,14,14,6) -> (None,10,10,16).
model.add(Conv2D(16, kernel_size=(5, 5), strides=(1, 1), activation='tanh', padding='valid'))
# P2: (None,10,10,16) -> (None,5,5,16).
model.add(AveragePooling2D(pool_size=(2, 2), strides=(2, 2), padding='valid'))
# Flatten: (None,5,5,16) -> (None, 400). 5*5*16=400
model.add(Flatten())
# FC1: (None, 400) -> (None,120).
model.add(Dense(120, activation='tanh'))
# FC2: (None,120) -> (None,84).
model.add(Dense(84, activation='tanh'))
# FC3: (None,84) -> (None,10).
model.add(Dense(10, activation='softmax'))
"""
optimizer优化器算法：更新模型参数。sgd / adma
loss损失函数：评估模型的正确率，最小化该函数，往预测正确的方向训练。categorical_crossentropy交叉熵损失函数
metrics指标：监视训练和测试步数，accuracy=图片被正确分类的比例
"""
# Compile the model
model.compile(loss='sparse_categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])
return model
model = LeNet_5()
model.summary()
from tensorflow.keras.utils import plot_model
# https://blog.csdn.net/weixin_36407399/article/details/87890230
plot_model(model, to_file='model.png', show_shapes=True)
WARNING:tensorflow:From /tensorflow-1.15.0/python3.6/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 28, 28, 6) 156
_________________________________________________________________
average_pooling2d (AveragePo (None, 14, 14, 6) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 10, 10, 16) 2416
_________________________________________________________________
average_pooling2d_1 (Average (None, 5, 5, 16) 0
_________________________________________________________________
flatten (Flatten) (None, 400) 0
_________________________________________________________________
dense (Dense) (None, 120) 48120
_________________________________________________________________
dense_1 (Dense) (None, 84) 10164
_________________________________________________________________
dense_2 (Dense) (None, 10) 850
=================================================================
Total params: 61,706
Trainable params: 61,706
Non-trainable params: 0
_________________________________________________________________
import time
t1=time.time()
# //batch_size=1024
history = model.fit(X_train, y_train, validation_data=(X_val, y_val),batch_size=1024,epochs=20)
t2=time.time()
print("模型训练时间：",t2-t1)
model.evaluate(X_test,y_test)
# Save the model.
model.save("lenet5_model.h5")
model.evaluate(X_test,y_test)
Train on 55000 samples, validate on 5000 samples
Epoch 1/20
55000/55000 [==============================] - 2s 39us/sample - loss: 2.2344 - acc: 0.2299 - val_loss: 2.1135 - val_acc: 0.3930
Epoch 2/20
55000/55000 [==============================] - 1s 10us/sample - loss: 1.9584 - acc: 0.4827 - val_loss: 1.7373 - val_acc: 0.6044
Epoch 3/20
55000/55000 [==============================] - 1s 10us/sample - loss: 1.5616 - acc: 0.6467 - val_loss: 1.3159 - val_acc: 0.7430
Epoch 4/20
55000/55000 [==============================] - 1s 10us/sample - loss: 1.2134 - acc: 0.7407 - val_loss: 1.0095 - val_acc: 0.8108
Epoch 5/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.9763 - acc: 0.7865 - val_loss: 0.8111 - val_acc: 0.8438
Epoch 6/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.8227 - acc: 0.8143 - val_loss: 0.6820 - val_acc: 0.8646
Epoch 7/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.7207 - acc: 0.8309 - val_loss: 0.5953 - val_acc: 0.8762
Epoch 8/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.6501 - acc: 0.8427 - val_loss: 0.5340 - val_acc: 0.8876
Epoch 9/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.5986 - acc: 0.8507 - val_loss: 0.4889 - val_acc: 0.8930
Epoch 10/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.5594 - acc: 0.8580 - val_loss: 0.4543 - val_acc: 0.8964
Epoch 11/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.5284 - acc: 0.8634 - val_loss: 0.4268 - val_acc: 0.8988
Epoch 12/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.5031 - acc: 0.8676 - val_loss: 0.4045 - val_acc: 0.9032
Epoch 13/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.4819 - acc: 0.8714 - val_loss: 0.3857 - val_acc: 0.9068
Epoch 14/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.4638 - acc: 0.8751 - val_loss: 0.3698 - val_acc: 0.9090
Epoch 15/20
55000/55000 [==============================] - 1s 9us/sample - loss: 0.4482 - acc: 0.8783 - val_loss: 0.3559 - val_acc: 0.9116
Epoch 16/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.4344 - acc: 0.8810 - val_loss: 0.3438 - val_acc: 0.9132
Epoch 17/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.4221 - acc: 0.8837 - val_loss: 0.3332 - val_acc: 0.9142
Epoch 18/20
55000/55000 [==============================] - 1s 10us/sample - loss: 0.4111 - acc: 0.8856 - val_loss: 0.3237 - val_acc: 0.9170
Epoch 19/20
55000/55000 [==============================] - 1s 9us/sample - loss: 0.4011 - acc: 0.8879 - val_loss: 0.3151 - val_acc: 0.9186
Epoch 20/20
55000/55000 [==============================] - 1s 11us/sample - loss: 0.3920 - acc: 0.8898 - val_loss: 0.3072 - val_acc: 0.9200
模型训练时间： 13.447073459625244
10000/10000 [==============================] - 1s 64us/sample - loss: 0.3687 - acc: 0.8963
10000/10000 [==============================] - 1s 66us/sample - loss: 0.3687 - acc: 0.8963
[0.36870885945558546, 0.8963]
# 加载已经训练好的模型
model = tf.keras.models.load_model('lenet5_model.h5')
# 对测试集进行预测
predictions = model.predict(X_test)
# 对预测结果选取概率最大索引
y_pred = np.argmax(predictions, axis=1)
# 测试集精确度
print('测试集错误率: {}'.format(np.mean(y_pred == y_test)))
# 用模型预测绘制一些例子
print('\n正确分类:')
plot_example(X_test, y_test, y_pred)
print('\n错误分类:')
plot_example_errors(X_test, y_test, y_pred)
# 训练
print('\n超过20代的训练:')
plt.title('training error')
plt.ylabel('Cost')
plt.xlabel('epoch')
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.legend(['train loss', 'val loss'], loc='upper right')
plt.show()
测试集错误率: 0.8963
正确分类:
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-vu1YGHMb-1585057954266)(output_12_1.png)]
错误分类:
超过20代的训练:
