Manipulating Tensors¶

Day 3¶

As tensors are the building blocks of AI algorithms it is very crucial to learn about them and their various methods. In this notebook we'll discover manipulations that we can do with matrices

This has been my go to playlist to understand math behind methods we'll use in this chapter

In [2]:

import torch

import torch

/Users/krishnaglodha/anaconda3/envs/aidoc/lib/python3.10/site-packages/torch/nn/modules/transformer.py:20: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:84.)
  device: torch.device = torch.device(torch._C._get_default_device()),  # torch.device('cpu'),

In [13]:

sample_tensor = torch.Tensor([2,3,4,1])

sample_tensor = torch.Tensor([2,3,4,1])

In [14]:

# increase each value with specific number 
sample_tensor += 10
sample_tensor

# increase each value with specific number sample_tensor += 10 sample_tensor

Out[14]:

tensor([12., 13., 14., 11.])

In [15]:

# reduce each value with specific number 
sample_tensor -= 2
sample_tensor

# reduce each value with specific number sample_tensor -= 2 sample_tensor

Out[15]:

tensor([10., 11., 12.,  9.])

In [16]:

# multiply each value with specific number 
sample_tensor *= 2
sample_tensor

# multiply each value with specific number sample_tensor *= 2 sample_tensor

Out[16]:

tensor([20., 22., 24., 18.])

In [17]:

# divide each value with specific number 
sample_tensor /= 2
sample_tensor

# divide each value with specific number sample_tensor /= 2 sample_tensor

Out[17]:

tensor([10., 11., 12.,  9.])

In [18]:

# Element-wise matrix multiplication
sample_tensor * sample_tensor

# Element-wise matrix multiplication sample_tensor * sample_tensor

Out[18]:

tensor([100., 121., 144.,  81.])

Matrix multiplication requires the inner dimension of matrices to match and the resulting matrix will have shape of outer dimension

In [7]:

# Matrix multiplication 
MATRIX_1 = torch.rand(2, 3)
MATRIX_2 = torch.rand(3, 4)
print(MATRIX_1)
print(MATRIX_2)

torch.matmul(MATRIX_1, MATRIX_2)

# Matrix multiplication MATRIX_1 = torch.rand(2, 3) MATRIX_2 = torch.rand(3, 4) print(MATRIX_1) print(MATRIX_2) torch.matmul(MATRIX_1, MATRIX_2)

tensor([[0.0912, 0.8151, 0.2648],
        [0.8988, 0.3201, 0.5228]])
tensor([[0.6952, 0.2691, 0.6689, 0.7677],
        [0.1456, 0.9015, 0.2138, 0.1805],
        [0.5212, 0.1230, 0.4983, 0.7268]])

Out[7]:

tensor([[0.3202, 0.7919, 0.3672, 0.4096],
        [0.9439, 0.5947, 0.9301, 1.1278]])

Transpose allows us to flip the axes of matrix

i.e. (3,2) matrix will be converted in (2,3)

In [9]:

MATRIX_1

MATRIX_1

Out[9]:

tensor([[0.0912, 0.8151, 0.2648],
        [0.8988, 0.3201, 0.5228]])

In [10]:

MATRIX_1.T

MATRIX_1.T

Out[10]:

tensor([[0.0912, 0.8988],
        [0.8151, 0.3201],
        [0.2648, 0.5228]])

In [13]:

# find min in matrix
print(MATRIX_1.min())
# find position of min in matrix
print(MATRIX_1.argmin())
# find max in matrix
print(MATRIX_1.max())
# find position of max in matrix
print(MATRIX_1.argmax())
# find mean in matrix
print(MATRIX_1.mean())
# calculate sum of matrix
print(MATRIX_1.sum())

# find min in matrix print(MATRIX_1.min()) # find position of min in matrix print(MATRIX_1.argmin()) # find max in matrix print(MATRIX_1.max()) # find position of max in matrix print(MATRIX_1.argmax()) # find mean in matrix print(MATRIX_1.mean()) # calculate sum of matrix print(MATRIX_1.sum())

tensor(0.0912)
tensor(0)
tensor(0.8988)
tensor(3)
tensor(0.4855)
tensor(2.9127)

Reshape - Reshape allows us to change the shape of matrix. The only rule is the multiplication of original shape should match with multiplication of reshaped matrix shape.

e.g. - Matrix of shape (2,3) can be reshaped into

• (3,2)
• (1,6)
• (6,1)

In [28]:

MATRIX_1,MATRIX_1.shape

MATRIX_1,MATRIX_1.shape

Out[28]:

(tensor([[3.0000, 0.8151, 0.2648],
         [0.8988, 0.3201, 0.5228]]),
 torch.Size([2, 3]))

In [22]:

RESHAPED_MAT = MATRIX_1.reshape(1,6)
print(RESHAPED_MAT)

RESHAPED_MAT = MATRIX_1.reshape(1,6) print(RESHAPED_MAT)

tensor([[0.0912, 0.8151, 0.2648, 0.8988, 0.3201, 0.5228]])

In [20]:

MATRIX_1.reshape(6,1)

MATRIX_1.reshape(6,1)

Out[20]:

tensor([[0.0912],
        [0.8151],
        [0.2648],
        [0.8988],
        [0.3201],
        [0.5228]])

In [23]:

RESHAPED_MAT[0][0] = 3

RESHAPED_MAT[0][0] = 3

In [27]:

RESHAPED_MAT

RESHAPED_MAT

Out[27]:

tensor([[3.0000, 0.8151, 0.2648, 0.8988, 0.3201, 0.5228]])

In [26]:

MATRIX_1

MATRIX_1

Out[26]:

tensor([[3.0000, 0.8151, 0.2648],
        [0.8988, 0.3201, 0.5228]])

Stacking Matrices allows us to put matrix on top of each other ( vertical stack ) or by each other's side ( horizontal stack ). Rule for stacking is that all matrices must be of same dimension

In [43]:

zero_mat = torch.zeros(3,2)
one_mat = torch.ones(3, 2) 
zero_mat.shape, one_mat.shape

zero_mat = torch.zeros(3,2) one_mat = torch.ones(3, 2) zero_mat.shape, one_mat.shape

Out[43]:

(torch.Size([3, 2]), torch.Size([3, 2]))

In [46]:

vstacked = torch.vstack([zero_mat, one_mat])

vstacked,vstacked.shape

vstacked = torch.vstack([zero_mat, one_mat]) vstacked,vstacked.shape

Out[46]:

(tensor([[0., 0.],
         [0., 0.],
         [0., 0.],
         [1., 1.],
         [1., 1.],
         [1., 1.]]),
 torch.Size([6, 2]))

In [48]:

hstacked = torch.hstack([zero_mat, one_mat])
hstacked,hstacked.shape

hstacked = torch.hstack([zero_mat, one_mat]) hstacked,hstacked.shape

Out[48]:

(tensor([[0., 0., 1., 1.],
         [0., 0., 1., 1.],
         [0., 0., 1., 1.]]),
 torch.Size([3, 4]))

squeeze allows us to remove all single dimensions from matrix

In [60]:

tens = torch.zeros(1,3,2,1)
tens

tens = torch.zeros(1,3,2,1) tens

Out[60]:

tensor([[[[0.],
          [0.]],

         [[0.],
          [0.]],

         [[0.],
          [0.]]]])

In [61]:

no_one= torch.squeeze(tens)
no_one

no_one= torch.squeeze(tens) no_one

Out[61]:

tensor([[0., 0.],
        [0., 0.],
        [0., 0.]])

unsqueeze allows us to add single dimension at specific dim in matrix

In [62]:

no_one, no_one.shape

no_one, no_one.shape

Out[62]:

(tensor([[0., 0.],
         [0., 0.],
         [0., 0.]]),
 torch.Size([3, 2]))

In [65]:

adding_one = torch.unsqueeze(no_one, dim=0)
adding_one,adding_one.shape

adding_one = torch.unsqueeze(no_one, dim=0) adding_one,adding_one.shape

Out[65]:

(tensor([[[0., 0.]],
 
         [[0., 0.]],
 
         [[0., 0.]]]),
 torch.Size([3, 1, 2]))

In [66]:

adding_one = torch.unsqueeze(no_one, dim=1)
adding_one,adding_one.shape

adding_one = torch.unsqueeze(no_one, dim=1) adding_one,adding_one.shape

Out[66]:

(tensor([[[0., 0.],
          [0., 0.],
          [0., 0.]]]),
 torch.Size([1, 3, 2]))

In [67]:

adding_one = torch.unsqueeze(no_one, dim=2)
adding_one,adding_one.shape

adding_one = torch.unsqueeze(no_one, dim=2) adding_one,adding_one.shape

Out[67]:

(tensor([[[0.],
          [0.]],
 
         [[0.],
          [0.]],
 
         [[0.],
          [0.]]]),
 torch.Size([3, 2, 1]))

permute allows us to rearrange the dimensions of matrix based on the index number of dimesion.

e.g. If you have a matrix of shape (2,3,4,1) and you want new matrix to be of shape (4,1,3,2) then you'll be putting permute for index numbers as (2,3,1,0) remember that these are not the dimension numbers but the index numbers of original dimensions

In [71]:

original_tensor = torch.rand(2,3,4,1)
original_tensor,original_tensor.shape

original_tensor = torch.rand(2,3,4,1) original_tensor,original_tensor.shape

Out[71]:

(tensor([[[[0.4067],
           [0.7352],
           [0.3255],
           [0.2045]],
 
          [[0.9771],
           [0.6874],
           [0.9577],
           [0.5992]],
 
          [[0.3105],
           [0.2828],
           [0.9761],
           [0.0919]]],
 
 
         [[[0.4163],
           [0.2669],
           [0.4095],
           [0.1668]],
 
          [[0.0218],
           [0.2487],
           [0.1014],
           [0.8136]],
 
          [[0.8378],
           [0.0290],
           [0.1106],
           [0.5918]]]]),
 torch.Size([2, 3, 4, 1]))

In [73]:

permuted_tensor = torch.permute(original_tensor,(2,0,3,1))
permuted_tensor,permuted_tensor.shape

permuted_tensor = torch.permute(original_tensor,(2,0,3,1)) permuted_tensor,permuted_tensor.shape

Out[73]:

(tensor([[[[0.4067, 0.9771, 0.3105]],
 
          [[0.4163, 0.0218, 0.8378]]],
 
 
         [[[0.7352, 0.6874, 0.2828]],
 
          [[0.2669, 0.2487, 0.0290]]],
 
 
         [[[0.3255, 0.9577, 0.9761]],
 
          [[0.4095, 0.1014, 0.1106]]],
 
 
         [[[0.2045, 0.5992, 0.0919]],
 
          [[0.1668, 0.8136, 0.5918]]]]),
 torch.Size([4, 2, 1, 3]))

In [10]:

import torch
import numpy as np

import torch import numpy as np

In [11]:

# converting numpy array to `tensor`
np_array = np.array([[1, 2, 3],[31, 23, 53]])
np_tensor = torch.from_numpy(np_array)
np_array,np_tensor

# converting numpy array to `tensor` np_array = np.array([[1, 2, 3],[31, 23, 53]]) np_tensor = torch.from_numpy(np_array) np_array,np_tensor

Out[11]:

(array([[ 1,  2,  3],
        [31, 23, 53]]),
 tensor([[ 1,  2,  3],
         [31, 23, 53]]))

In [12]:

# by default datatype for tensor converted from numpy array is float64 | int64
np_tensor.dtype

# by default datatype for tensor converted from numpy array is float64 | int64 np_tensor.dtype

Out[12]:

torch.int64

In [13]:

# once np array is converted in tensor, changing the np array will not change the tensor
np_array = np.array([31, 23, 53])
np_array,np_tensor

# once np array is converted in tensor, changing the np array will not change the tensor np_array = np.array([31, 23, 53]) np_array,np_tensor

Out[13]:

(array([31, 23, 53]),
 tensor([[ 1,  2,  3],
         [31, 23, 53]]))

In [14]:

# once np array is converted in tensor, manipulating the np array will change the tensor
np_array = np.array([[1, 2, 3],[31, 23, 53]])
np_tensor = torch.from_numpy(np_array)
np_array,np_tensor

# once np array is converted in tensor, manipulating the np array will change the tensor np_array = np.array([[1, 2, 3],[31, 23, 53]]) np_tensor = torch.from_numpy(np_array) np_array,np_tensor

Out[14]:

(array([[ 1,  2,  3],
        [31, 23, 53]]),
 tensor([[ 1,  2,  3],
         [31, 23, 53]]))

In [15]:

np_array *= 4
np_array,np_tensor

np_array *= 4 np_array,np_tensor

Out[15]:

(array([[  4,   8,  12],
        [124,  92, 212]]),
 tensor([[  4,   8,  12],
         [124,  92, 212]]))

In [16]:

# converting `tensor` to numpy
og_tensor = torch.rand(2, 3)
tensor_to_numpy = og_tensor.numpy()
og_tensor, tensor_to_numpy

# converting `tensor` to numpy og_tensor = torch.rand(2, 3) tensor_to_numpy = og_tensor.numpy() og_tensor, tensor_to_numpy

Out[16]:

(tensor([[0.9867, 0.8871, 0.8978],
         [0.3541, 0.4153, 0.9079]]),
 array([[0.9866963 , 0.88708484, 0.8977984 ],
        [0.35412902, 0.41526848, 0.90792847]], dtype=float32))

Little bit about generating random numbers.¶

Random numbers that we generate are generally truly random, that means if we generate a tensor with tensor.rand() and if we hit it again, we might get totally different results

In [17]:

# generate  random tensor

RANDOM_T = torch.rand(3, 4)
RANDOM_T

# generate random tensor RANDOM_T = torch.rand(3, 4) RANDOM_T

Out[17]:

tensor([[0.8340, 0.6770, 0.5381, 0.6980],
        [0.4639, 0.2720, 0.3251, 0.6900],
        [0.6232, 0.5145, 0.8836, 0.7499]])

If you keep hitting above code, every time you'll get different results

but

If you want to control this randomness, i.e. reproduce same random numbers again and again, you can do it with manual_seed

In [18]:

flavour = 71 # It can be any random number
torch.manual_seed(flavour) 
RANDOM_T = torch.rand(3, 4)
RANDOM_T

flavour = 71 # It can be any random number torch.manual_seed(flavour) RANDOM_T = torch.rand(3, 4) RANDOM_T

Out[18]:

tensor([[0.5073, 0.3347, 0.9694, 0.7767],
        [0.9654, 0.5069, 0.8665, 0.3994],
        [0.6388, 0.4128, 0.8075, 0.6319]])