pytorc 加载数据

数据加载

  • 加载预制数据
  • 加载本地数据

加载预制数据

步骤:
(1). cifarSet

cifarSet = torchvision.datasets.CIFAR10(root = "../data/cifar/", train= True, download = True, transform = mytransform )

(2). cifarLoader

cifarLoader = torch.utils.data.DataLoader(cifarSet, batch_size= 10, shuffle= False, num_workers= 2)

综合:

mytransform = transforms.Compose([
    transforms.ToTensor()
    ]
)
cifarSet = torchvision.datasets.CIFAR10(root = "../data/cifar/", train= True, download = True, transform = mytransform )
cifarLoader = torch.utils.data.DataLoader(cifarSet, batch_size= 10, shuffle= False, num_workers= 2)
### cifar10数据集(Example)
1) cifar10数据下载, 并进行简单的transform, 包括data、target
In [2]: import torchvision

In [3]: cifarSet = torchvision.datasets.CIFAR10(root = "./", train= True, download = True)
Files already downloaded and verified

In [4]: cifarSet
Out[4]: 
Dataset CIFAR10
    Number of datapoints: 50000
    Split: train
    Root Location: ./
    Transforms (if any): None
    Target Transforms (if any): None

加载时可选参数

In [9]: cifarSet = torchvision.datasets.CIFAR10?
Init signature: torchvision.datasets.CIFAR10(root, train=True, transform=None, target_transform=None, download=False)
Docstring:     
`CIFAR10 <https://www.cs.toronto.edu/~kriz/cifar.html>`_ Dataset.

Args:
    root (string): Root directory of dataset where directory
        ``cifar-10-batches-py`` exists or will be saved to if download is set to True.
    train (bool, optional): If True, creates dataset from training set, otherwise
        creates from test set.
    transform (callable, optional): A function/transform that  takes in an PIL image
        and returns a transformed version. E.g, ``transforms.RandomCrop``
    target_transform (callable, optional): A function/transform that takes in the
        target and transforms it.
    download (bool, optional): If true, downloads the dataset from the internet and
        puts it in root directory. If dataset is already downloaded, it is not
        downloaded again.
File:           ~/.conda/envs/inb/lib/python3.6/site-packages/torchvision/datasets/cifar.py
Type:           type

在构造函数中,不同的数据集直接的构造函数会有些许不同,但是他们共同拥有 keyword 参数。 In the constructor, each dataset has a slightly different API as needed, but they all take the keyword args: - transform: 一个函数,原始图片作为输入,返回一个转换后的图片。(详情请看下面关于torchvision-tranform的部分)

target_transform - 一个函数,输入为target,输出对其的转换。例子,输入的是图片标注的string,输出为word的索引

  1. 训练数据引入
    torch.utils.data.Dataset
    文档

加载本地数据

步骤:
(1). Mydataset===>自制 类似于cifarset
详见

class MyDataset():
	def __init__():
	def __getitem__():
	def __len__():
mydataset = MyDataset()

(2). cifarLoader

cifarLoader = torch.utils.data.DataLoader(mydataset, batch_size= 10, shuffle= False, num_workers= 2)

实例讲解

你可能感兴趣的:(应用,理论,pytorch入门与实践)