# 深度学习实验二

NNDL 实验二 pytorch入门

1.张量：在深度学习框架中，数据经常用张量(Tensor)的形式来存储。张量是矩阵的扩展与延伸，可以认为是高阶的矩阵，是标量,向量,矩阵的高维拓展。
2.算子：算子是构建复杂机器学习模型的基础组件，包含一个函数f(x)的前向函数和反向函数。

1.2 张量
1.2.1 创建张量

1.2.1.1 指定数据创建张量

（1）通过指定的Python列表数据[2.0, 3.0, 4.0]，创建一个一维张量。

``````#导入torch
import torch
ndim_1_Tensor = torch.tensor([2.0, 3.0, 4.0])
print(ndim_1_Tensor)
``````

``````tensor([2., 3., 4.])
``````

（2）通过指定的Python列表数据来创建类似矩阵（matrix）的二维张量。

``````ndim_2_Tensor = torch.tensor([[1.0, 2.0, 3.0],[4.0, 5.0, 6.0]])
print(ndim_2_Tensor)
``````

``````tensor([[1., 2., 3.],
[4., 5., 6.]])
``````

（3）同样地，还可以创建维度为3、4…N等更复杂的多维张量。

``````ndim_3_Tensor = torch.tensor([[[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]], [[11, 12, 13, 14, 15], [16, 17, 18, 19, 20]]])
print(ndim_3_Tensor)
``````

``````tensor([[[ 1,  2,  3,  4,  5],
[ 6,  7,  8,  9, 10]],
[[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20]]])
``````

``````ndim_2_Tensor = torch.tensor([[1.0, 2.0],[4.0, 5.0, 6.0]])
``````

``````    ndim_2_Tensor = torch.tensor([[1.0, 2.0],[4.0, 5.0, 6.0]])
ValueError: expected sequence of length 2 at dim 1 (got 3)
``````

1.2.1.2 指定形状创建

``````m=2
n=3
zeros_Tensor = torch.zeros([m, n])
ones_Tensor = torch.ones([m, n])
full_Tensor = torch.full([m, n], 10)
print('zeros Tensor: ', zeros_Tensor)
print('ones Tensor: ', ones_Tensor)
print('full Tensor: ', full_Tensor)
``````

``````zeros Tensor:  tensor([[0., 0., 0.],
[0., 0., 0.]])
ones Tensor:  tensor([[1., 1., 1.],
[1., 1., 1.]])
full Tensor:  tensor([[10., 10., 10.],
[10., 10., 10.]])
``````

1.2.1.3 指定区间创建

``````arange_Tensor = torch.arange(start=1, end=5, step=1)
print('arange_Tensor:', arange_Tensor)
linspace_Tensor = torch.linspace(start=1, end=5, steps=5)
print('linspace_Tensor:', linspace_Tensor)
``````

``````arange_Tensor: tensor([1, 2, 3, 4])
linspace_Tensor: tensor([1., 2., 3., 4., 5.])
``````

1.2.2 张量的属性

1.2.2.1 张量的形状

``````ndim_4_Tensor = torch.ones([2, 3, 4, 5])
print("Number of dimensions:", ndim_4_Tensor.ndim)
print("Shape of Tensor:", ndim_4_Tensor.shape)
print("Elements number along axis 0 of Tensor:", ndim_4_Tensor.shape[0])
print("Elements number along the last axis of Tensor:", ndim_4_Tensor.shape[-1])
print('Number of elements in Tensor: ', ndim_4_Tensor.size)

``````

``````Number of dimensions: 4
Shape of Tensor: torch.Size([2, 3, 4, 5])
Elements number along axis 0 of Tensor: 2
Elements number along the last axis of Tensor: 5
Number of elements in Tensor:  <built-in method size of Tensor object at 0x000001E83F173840>
``````

1.2.2.2 形状的改变

``````# 定义一个shape为[3,2,5]的三维Tensor
ndim_3_Tensor = torch.tensor([[[1, 2, 3, 4, 5],
[6, 7, 8, 9, 10]],
[[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20]],
[[21, 22, 23, 24, 25],
[26, 27, 28, 29, 30]]])
print("the shape of ndim_3_Tensor:", ndim_3_Tensor.shape)
reshape_Tensor = torch.reshape(ndim_3_Tensor, [2, 5, 3])
print("After reshape:", reshape_Tensor)
``````

``````the shape of ndim_3_Tensor: torch.Size([3, 2, 5])
After reshape: tensor([[[ 1,  2,  3],
[ 4,  5,  6],
[ 7,  8,  9],
[10, 11, 12],
[13, 14, 15]],

[[16, 17, 18],
[19, 20, 21],
[22, 23, 24],
[25, 26, 27],
[28, 29, 30]]])
``````

-1表示这个维度的值是从张量的元素总数和剩余维度推断出来的。因此，有且只有一个维度可以被设置为-1。
0表示实际的维数是从张量的对应维数中复制出来的，因此shape中0所对应的索引值不能超过张量的总维度。

``````new_Tensor1 = ndim_3_Tensor.reshape([-1])
print('new Tensor 1 shape: ', new_Tensor1.shape)
new_Tensor2 = ndim_3_Tensor.reshape([0, 5, 2])
print('new Tensor 2 shape: ', new_Tensor2.shape)
``````

``````new Tensor 1 shape:  [30]
new Tensor 2 shape:  [3, 5, 2]
``````

1.2.2.3 张量的数据类型

``````print("Tensor dtype from Python integers:", torch.tensor(1).dtype)
print("Tensor dtype from Python floating point:", torch.tensor(1.0).dtype)
``````

``````Tensor dtype from Python integers: torch.int64
Tensor dtype from Python floating point: torch.float32
``````

``````# 定义dtype为float32的Tensor
float32_Tensor = torch.tensor(1.0)
# torch.cast可以将输入数据的数据类型转换为指定的dtype并输出。支持输出和输入数据类型相同。
int64_Tensor = float32_Tensor.to(torch.int64)
print('Tensor after cast to int64:', int64_Tensor.dtype)
``````

``````Tensor after cast to int64: torch.int64
``````

1.2.2.4 张量的设备位置

``````# 创建CPU上的Tensor
cpu_Tensor = torch.tensor(1, device=torch.device('cpu'))
print('cpu_Tensor:', cpu_Tensor.device)
# 创建GPU上的Tensor
gpu_Tensor = torch.tensor(1, device=torch.device('gpu'))
print('gpu_Tensor:', gpu_Tensor.device)
# 创建固定内存上的Tensor
pin_memory_Tensor = torch.tensor(1, device=torch.device('cudaPinned'))
print('pin memory Tensor:', pin_memory_Tensor.device)
``````

``````cpu Tensor:  CPUPlace
gpu Tensor:  CUDAPlace(0)
pin memory Tensor:  CUDAPinnedPlace
``````

1.2.3 张量与Numpy数组转换

``````ndim_1_Tensor =torch.tensor([1., 2.])
# 将当前 Tensor 转化为 numpy.ndarray
print('Tensor to convert: ', ndim_1_Tensor.numpy())
``````

``````Tensor to convert:  [1. 2.]
``````

1.2.4 张量的访问

1.2.4.1 索引和切片

1.2.4.2 访问张量

``````# 定义1个一维Tensor
ndim_1_Tensor = torch.tensor([0, 1, 2, 3, 4, 5, 6, 7, 8])
print("Origin Tensor:", ndim_1_Tensor)
print("First element:", ndim_1_Tensor[0])
print("Last element:", ndim_1_Tensor[-1])
print("All element:", ndim_1_Tensor[:])
print("Before 3:", ndim_1_Tensor[:3])
print("Interval of 3:", ndim_1_Tensor[::3])
print("Reverse:", ndim_1_Tensor.flip(-1))
``````

``````Origin Tensor: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8])
First element: tensor(0)
Last element: tensor(8)
All element: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8])
Before 3: tensor([0, 1, 2])
Interval of 3: tensor([0, 3, 6])
Reverse: tensor([8, 7, 6, 5, 4, 3, 2, 1, 0])
``````

1.2.4.3 修改张量

``````ndim_2_Tensor = torch.ones([2, 3])
ndim_2_Tensor = ndim_2_Tensor.to(torch.float32)
print('Origin Tensor: ', ndim_2_Tensor)
# 修改第1维为0
ndim_2_Tensor[0] = 0
print('change Tensor: ', ndim_2_Tensor)
# 修改第1维为2.1
ndim_2_Tensor[0:1] = 2.1
print('change Tensor: ', ndim_2_Tensor)
# 修改全部Tensor
ndim_2_Tensor[...] = 3
print('change Tensor: ', ndim_2_Tensor)
``````

``````Origin Tensor:  tensor([[1., 1., 1.],
[1., 1., 1.]])
change Tensor:  tensor([[0., 0., 0.],
[1., 1., 1.]])
change Tensor:  tensor([[2.1000, 2.1000, 2.1000],
[1.0000, 1.0000, 1.0000]])
change Tensor:  tensor([[3., 3., 3.],
[3., 3., 3.]])
``````

1.2.5 张量的运算

``````# 定义两个Tensor
x = torch.tensor([[1.1, 2.2], [3.3, 4.4]])
x = x.to(torch.float64)
y = torch.tensor([[5.5, 6.6], [7.7, 8.8]])
y = y.to(torch.float64)
# 第二种调用方法
``````

``````Method 1:  tensor([[ 6.6000,  8.8000],
[11.0000, 13.2000]], dtype=torch.float64)
Method 2:  tensor([[ 6.6000,  8.8000],
[11.0000, 13.2000]], dtype=torch.float64)
``````

1.2.5.1 数学运算

``````x.abs()                       # 逐元素取绝对值
x.ceil()                      # 逐元素向上取整
x.floor()                     # 逐元素向下取整
x.round()                     # 逐元素四舍五入
x.exp()                       # 逐元素计算自然常数为底的指数
x.log()                       # 逐元素计算x的自然对数
x.reciprocal()                # 逐元素求倒数
x.square()                    # 逐元素计算平方
x.sqrt()                      # 逐元素计算平方根
x.sin()                       # 逐元素计算正弦
x.cos()                       # 逐元素计算余弦
x.subtract(y)                 # 逐元素减
x.multiply(y)                 # 逐元素乘（积）
x.divide(y)                   # 逐元素除
x.mod(y)                      # 逐元素除并取余
x.pow(y)                      # 逐元素幂
x.max()                       # 指定维度上元素最大值，默认为全部维度
x.min()                       # 指定维度上元素最小值，默认为全部维度
x.prod()                      # 指定维度上元素累乘，默认为全部维度
x.sum()                       # 指定维度上元素的和，默认为全部维度
``````

``````x + y  -> x.add(y)            # 逐元素加
x - y  -> x.subtract(y)       # 逐元素减
x * y  -> x.multiply(y)       # 逐元素乘（积）
x / y  -> x.divide(y)         # 逐元素除
x % y  -> x.mod(y)            # 逐元素除并取余
x ** y -> x.pow(y)            # 逐元素幂
``````

1.2.5.2 逻辑运算

``````x.isfinite()                  # 判断Tensor中元素是否是有限的数字，即不包括inf与nan
x.equal_all(y)                # 判断两个Tensor的全部元素是否相等，并返回形状为[1]的布尔类Tensor
x.equal(y)                    # 判断两个Tensor的每个元素是否相等，并返回形状相同的布尔类Tensor
x.not_equal(y)                # 判断两个Tensor的每个元素是否不相等
x.less_than(y)                # 判断Tensor x的元素是否小于Tensor y的对应元素
x.less_equal(y)               # 判断Tensor x的元素是否小于或等于Tensor y的对应元素
x.greater_than(y)             # 判断Tensor x的元素是否大于Tensor y的对应元素
x.greater_equal(y)            # 判断Tensor x的元素是否大于或等于Tensor y的对应元素
x.allclose(y)                 # 判断两个Tensor的全部元素是否接近
``````

1.2.5.3 矩阵运算

``````x.t()                         # 矩阵转置
x.transpose([1, 0])           # 交换第 0 维与第 1 维的顺序
x.norm('fro')                 # 矩阵的弗罗贝尼乌斯范数
x.dist(y, p=2)                # 矩阵（x-y）的2范数
x.matmul(y)                   # 矩阵乘法
``````

1.2.5.4 广播机制

1）每个张量至少为一维张量。
2）从后往前比较张量的形状，当前维度的大小要么相等，要么其中一个等于1，要么其中一个不存在。

``````# 当两个Tensor的形状一致时，可以广播
x = torch.ones((2, 3, 4))
y = torch.ones((2, 3, 4))
z = x + y
print('broadcasting with two same shape tensor: ', z.shape)
x = torch.ones((2, 3, 1, 5))
y = torch.ones((3, 4, 1))
# 从后往前依次比较：
# 第一次：y的维度大小是1
# 第二次：x的维度大小是1
# 第三次：x和y的维度大小相等，都为3
# 第四次：y的维度不存在
# 所以x和y是可以广播的
z = x + y
print('broadcasting with two different shape tensor:', z.shape)
``````

``````broadcasting with two same shape tensor:  torch.Size([2, 3, 4])
broadcasting with two different shape tensor: torch.Size([2, 3, 4, 5])
``````

``````x = torch.ones((2, 3, 4))
y = torch.ones((2, 3, 6))
z = x + y
``````

``````RuntimeError: The size of tensor a (4) must match the size of tensor b (6) at non-singleton dimension 2
``````

2.2.1. 读取数据集

``````import os

os.makedirs(os.path.join('..', 'data'), exist_ok=True)
data_file = os.path.join('..', 'data', 'house_tiny.csv')
with open(data_file, 'w') as f:
f.write('NumRooms,Alley,Pricen')  # 列名
f.write('NA,Pave,127500n')  # 每行表示一个数据样本
f.write('2,NA,106000n')
f.write('4,NA,178100n')
f.write('NA,NA,140000n')
``````

``````# 如果没有安装pandas，只需取消对以下行的注释来安装pandas
# !pip install pandas
import pandas as pd

print(data)
``````

``````   NumRooms Alley   Price
0       NaN  Pave  127500
1       2.0   NaN  106000
2       4.0   NaN  178100
3       NaN   NaN  140000
``````

2.2.2. 处理缺失值

``````inputs, outputs = data.iloc[:, 0:2], data.iloc[:, 2]
inputs = inputs.fillna(inputs.mean())
print(inputs)
``````

``````   NumRooms Alley
0       3.0  Pave
1       2.0   NaN
2       4.0   NaN
3       3.0   NaN
``````

``````inputs = pd.get_dummies(inputs, dummy_na=True)
print(inputs)
``````

``````   NumRooms  Alley_Pave  Alley_nan
0       3.0           1          0
1       2.0           0          1
2       4.0           0          1
3       3.0           0          1
``````

2.2.3. 转换为张量格式

``````import torch
X, y = torch.tensor(inputs.values), torch.tensor(outputs.values)
X, y
``````

``````(tensor([[3., 1., 0.],
[2., 0., 1.],
[4., 0., 1.],
[3., 0., 1.]], dtype=torch.float64),
tensor([127500, 106000, 178100, 140000]))
``````

pandas软件包是Python中常用的数据分析工具中，pandas可以与张量兼容。