torch.nn模块
本文就根据官网文档整理一下torch.nn模块
TORCH.NN非常重要，也是以后用pytorch搭建神经网络的关键模块。所以值得认真学习。
TORCH.NN
第一个关键类：
CLASS torch.nn.Parameter
A kind of Tensor that is to be considered a module parameter.
Parameters are
Tensorsubclasses, that have a very special property when used withModules - when they’re assigned as Module attributes they are automatically added to the list of its parameters, and will appear e.g. in parameters() iterator. Assigning a Tensor doesn’t have such effect. This is because one might want to cache some temporary state, like last hidden state of the RNN, in the model. If there was no such class as Parameter, these temporaries would get registered too.
简单来说，就是Tensor类的特殊化实例，利用torch.nn.Parameter(data,required_grad)定义的变量可以直接作为网络模型的parameter
需要的参数有：
- data (Tensor) – parameter tensor.
- requires_grad (bool, optional) – Default: True
第二个关键类
CLASS torch.nn.Module
Base class for all neural network modules.
Your models should also subclass this class.
Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes:
很关键很重要的搭建神经网络的父类
举例：
import torch.nn as nn
import torch.nn.functional as F
class Model(nn.Module):
def __init__(self):
super(Model, self).__init__()
self.conv1 = nn.Conv2d(1, 20, 5)
self.conv2 = nn.Conv2d(20, 20, 5)
def forward(self, x):
x = F.relu(self.conv1(x))
return F.relu(self.conv2(x))
那么该方法都包括哪些方法呢：
add_module(name, module)
Adds a child module to the current module. The module can be
accessed as an attribute using the given name. parameter:
- name (string) – name of the child module. The child module can be accessed from this module using the given name
- module (Module) – child module to be added to the module.
作用：添加子模型
—————————————————————————————
apply(fn)
‘’‘
Parameters：fn (Module -> None) – function to be applied to each submodule
Return：self
Return type:Module
’‘’
Applies
fnrecursively to every submodule (as returned by.children()) as well as self. Typical use includes initializing the parameters of a model (see also torch-nn-init).
举例：
>>> def init_weights(m):
>>> print(m)
>>> if type(m) == nn.Linear:
>>> m.weight.data.fill_(1.0)
>>> print(m.weight)
>>> net = nn.Sequential(nn.Linear(2, 2), nn.Linear(2, 2))
>>> net.apply(init_weights)
相当于对net内的每一个module都进行一次fn(init_weights)处理“
结果：
Linear(in_features=2, out_features=2, bias=True)
Parameter containing:
tensor([[ 1., 1.],
[ 1., 1.]])
Linear(in_features=2, out_features=2, bias=True)
Parameter containing:
tensor([[ 1., 1.],
[ 1., 1.]])
Sequential(
(0): Linear(in_features=2, out_features=2, bias=True)
(1): Linear(in_features=2, out_features=2, bias=True)
)
Sequential(
(0): Linear(in_features=2, out_features=2, bias=True)
(1): Linear(in_features=2, out_features=2, bias=True)
)
—————————————————————————————
cpu()
Moves all model parameters and buffers to the CPU.
Returns ：self
Return type：Module
—————————————————————————————
cuda(device=None)
Moves all model parameters and buffers to the GPU.
This also makes associated parameters and buffers different objects. So it should be called before constructing optimizer if the module will live on GPU while being optimized.
Parameters : device (int, optional) – if specified, all parameters will be copied to that device
Returns :self
Return type :Module
—————————————————————————————
double() / float() /half()
Casts all floating point parameters and buffers to double datatype.
Returns:self
Return type:Module
—————————————————————————————
关键
forward(*input)
Defines the computation performed at every call.Should be overridden by all subclasses.定义自定义神经网络时的前向传播运算
—————————————————————————————
modules()
Returns an iterator over all modules in the network.
Yields:Module – a module in the network
注：Duplicate modules are returned only once. In the following example, l will be returned only once.
就是重复的结构之返回一次
例子：
>>> l = nn.Linear(2, 2)
>>> net = nn.Sequential(l, l)
>>> for idx, m in enumerate(net.modules()):
print(idx, '->', m)
0 -> Sequential(
(0): Linear(in_features=2, out_features=2, bias=True)
(1): Linear(in_features=2, out_features=2, bias=True)
)
1 -> Linear(in_features=2, out_features=2, bias=True)
—————————————————————————————
named_buffers(prefix='', recurse=True)
named_children()
named_modules(memo=None, prefix='')
named_parameters(prefix='', recurse=True)
（以named_parameters为例子）
Returns an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.
就是会返回名字+parameter的迭代形式
即:
Yields：(string, Parameter) – Tuple containing the name and parameter
举例：
>>> for name, param in self.named_parameters():
>>> if name in ['bias']:
>>> print(param.size())
—————————————————————————————
parameters(recurse=True)
Returns an iterator over module parameters.
This is typically passed to an optimizer.
返回 Parameter – module parameter
参数：recurse (bool) – if True, then yields parameters of this module and all submodules. Otherwise, yields only parameters that are direct members of this module.
举例子：
>>> for param in model.parameters():
>>> print(type(param.data), param.size())
(20L,)
(20L, 1L, 5L, 5L)
—————————————————————————————
to(*args, **kwargs)
Moves and/or casts the parameters and buffers.
This can be called as
to(device=None, dtype=None, non_blocking=False)
to(dtype, non_blocking=False)
to(tensor, non_blocking=False)
Its signature is similar to torch.Tensor.to(), but only accepts floating point desired dtype s. In addition, this method will only cast the floating point parameters and buffers to dtype (if given). The integral parameters and buffers will be moved device, if that is given, but with dtypes unchanged. When non_blocking is set, it tries to convert/move asynchronously with respect to the host if possible, e.g., moving CPU Tensors with pinned memory to CUDA devices.
举例子：
>>> linear = nn.Linear(2, 2)
>>> linear.weight
Parameter containing:
tensor([[ 0.1913, -0.3420],
[-0.5113, -0.2325]])
>>> linear.to(torch.double)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1913, -0.3420],
[-0.5113, -0.2325]], dtype=torch.float64)
>>> gpu1 = torch.device("cuda:1")
>>> linear.to(gpu1, dtype=torch.half, non_blocking=True)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1914, -0.3420],
[-0.5112, -0.2324]], dtype=torch.float16, device='cuda:1')
>>> cpu = torch.device("cpu")
>>> linear.to(cpu)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1914, -0.3420],
[-0.5112, -0.2324]], dtype=torch.float16)
———————————————————————————————