PyTorch深度学习实践09——多分类问题（softmax分类器）

• 分类问题的输出应该是一个分布：即需要满足单个样本点概率≥0且样本点概率之和=1
• softmax layer：
• 损失函数：Cross Entropy
• Exercise9-1：CrossEntropyLoss和NLLLoss的区别

CrossEntropyLoss的输入不要求进行LogSoftmax运算，NLLLoss要求输入先进行LogSoftmax运算。

CrossEntropyLoss <==> LogSoftmax + NLLLoss。也就是说使用CrossEntropyLoss最后一层(线性层)是不需要做其他变化的；使用NLLLoss之前，需要对最后一层(线性层)先进行SoftMax处理，再进行log操作

• transform作用： 

1. 把PIL读入的图像（WxHxC）转换成Torch.Tensor（调换矩阵尺寸的顺序变为CxWxH）
2. 将pixel的数值映射到[0, 1]

• 在本节中，将train和test也封装成函数，使代码更加简洁

课上代码实现：

代码说明：

1. 上一讲中有提及from torchvision import datasets。该datasets里面init，getitem,len魔法函数已实现。
2. torch.max的返回值有两个，第一个是每一行的最大值是多少，第二个是每一行最大值的下标(索引)是多少。
3. .shape和.size()的区别
4. 代码中"_"的说明python中各种_的用法

import torch
# For constructing dataloader
from torchvision import datasets
from torchvision import transforms
from torch.utils.data import DataLoader

import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

#prepare dataset
batch_size = 64
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307, ), (0.3081, ))])

train_dataset = datasets.MNIST(root='./dataset/mnist/', train=True, download=True, transform=transform)
test_dataset = datasets.MNIST(root='./dataset/mnist/', train=False, download=True, transform=transform)
train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size)
test_loader = DataLoader(test_dataset, shuffle=True, batch_size=batch_size)

# design model
class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.l1 = nn.Linear(784, 512)
        self.l2 = nn.Linear(512, 256)
        self.l3 = nn.Linear(256, 128)
        self.l4 = nn.Linear(128, 64)
        self.l5 = nn.Linear(64, 10)

    def forward(self, x):
        x = x.view(-1, 784) # -1其实就是自动获取mini_batch
        x = F.relu(self.l1(x))
        x = F.relu(self.l2(x))
        x = F.relu(self.l3(x))
        x = F.relu(self.l4(x))
        x = self.l5(x)
        return x

model = Model()

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)

# training cycle forward, backward, update

def train(epoch):
    running_loss = 0.0
    for batch_idx, data in enumerate(train_loader, 0):
        # 获取一个批次的数据和标签
        inputs, target = data
        optimizer.zero_grad()

        # 获得模型预测结果 (64, 10)
        outputs = model(inputs)

        # 交叉熵代价函数outputs(64,10),target（64）
        loss = criterion(outputs, target)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()
        if batch_idx % 300 == 299:
            print('[epoch:%d, batch_idx:%5d] loss: %.3f' % (epoch + 1, batch_idx + 1, running_loss / 300))
            running_loss = 0.0
def test():
    correct = 0
    total = 0

    with torch.no_grad():
        for data in test_loader:
            inputs, labels = data
            outputs = model(inputs)
            _, predicted = torch.max(outputs.data, dim=1)
            total += labels.size(0)
            correct += (predicted == labels).sum().item()

    print('accuracy on test set: %d %% ' % (100*correct/total))


if __name__ == "__main__":
    for epoch in range(10):
        train(epoch)
        test()