Learning about Tensors¶

Day 2¶

It turns out Tensors are legos of AI world. As everything that we do is converted into numbers and lists, Tensors are responsible for managing those. Now this is where it's time to take first Maths lesson.

Math lesson - What are Tensors ?¶

Tensors are ways to store data in multi-dimensions. , very good representation of it is at here

Checkout few videos about Tensors here -

• English

• Hindi

In [2]:

import torch

import torch

/Users/krishnaglodha/anaconda3/envs/aidoc/lib/python3.10/site-packages/torch/nn/modules/transformer.py:20: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:84.)
  device: torch.device = torch.device(torch._C._get_default_device()),  # torch.device('cpu'),

Simplest representation of tensors is single number¶

In [14]:

scalar = torch.tensor(7)
scalar

scalar = torch.tensor(7) scalar

Out[14]:

tensor(7)

In [15]:

# see dimensions 
print(f"dimension of scalar is {scalar.ndim}")
# see shape
print(f"shape of scalar is {scalar.shape}")
# see item
print(f"item in scalar is {scalar.item()}")

# see dimensions print(f"dimension of scalar is {scalar.ndim}") # see shape print(f"shape of scalar is {scalar.shape}") # see item print(f"item in scalar is {scalar.item()}")

dimension of scalar is 0
shape of scalar is torch.Size([])
item in scalar is 7

Representation of list as tensors¶

In [16]:

vector = torch.tensor([1,2,4,5])
vector

vector = torch.tensor([1,2,4,5]) vector

Out[16]:

tensor([1, 2, 4, 5])

In [17]:

# see dimensions  - how many lists are there in the tensor
print(f"dimension of vector is {vector.ndim}")
# see shape - shape represents how many elements are there in the tensor list
print(f"shape of vector is {vector.shape}")
# see item -  Item can be only scalar, thus we went at 0th index of vector and got that item
print(f"item in vector is {vector[0].item()}")

# see dimensions - how many lists are there in the tensor print(f"dimension of vector is {vector.ndim}") # see shape - shape represents how many elements are there in the tensor list print(f"shape of vector is {vector.shape}") # see item - Item can be only scalar, thus we went at 0th index of vector and got that item print(f"item in vector is {vector[0].item()}")

dimension of vector is 1
shape of vector is torch.Size([4])
item in vector is 1

Representation of matrices as tensors¶

As you can see in case of matrix we have a Bigger list, which contains 2 smaller lists, and each of this small list contains equal number of items.

In [20]:

MATRIX = torch.tensor([[1,2,4,5],[3,4,2,9],[3,1,8,64]])
MATRIX

MATRIX = torch.tensor([[1,2,4,5],[3,4,2,9],[3,1,8,64]]) MATRIX

Out[20]:

tensor([[ 1,  2,  4,  5],
        [ 3,  4,  2,  9],
        [ 3,  1,  8, 64]])

In [26]:

# see dimensions  - how many lists are there in the tensor )
print(f"dimension of MATRIX is {MATRIX.ndim}")
# see shape - shape represents how many elements are there in the tensor list ( here we have 3 lists and each list has 4 elements)
print(f"shape of MATRIX is {MATRIX.shape}")
# see item -  Item can be only scalar, thus we went at 0th index of MATRIX , we got list there [1,2,4,5], thus we went inside that list and looked for 0th element and got that item
print(f"item in MATRIX is {MATRIX[0][0].item()}")

# see dimensions - how many lists are there in the tensor ) print(f"dimension of MATRIX is {MATRIX.ndim}") # see shape - shape represents how many elements are there in the tensor list ( here we have 3 lists and each list has 4 elements) print(f"shape of MATRIX is {MATRIX.shape}") # see item - Item can be only scalar, thus we went at 0th index of MATRIX , we got list there [1,2,4,5], thus we went inside that list and looked for 0th element and got that item print(f"item in MATRIX is {MATRIX[0][0].item()}")

dimension of MATRIX is 2
shape of MATRIX is torch.Size([3, 4])
item in MATRIX is 1

Representation of multi-dimension matrices as tensors¶

Complex data can be a combination of multiple matrices, which are known as tensors

As shown below , Tensor is a bigger list, which has 2 big lists , and each of there big lists have 3 smaller lists, and each of this smaller list has 2 values in them.

In [28]:

# TENSOR
TENSOR = torch.tensor([[[1,4],[3,1],[5,4]],
                        [[11,14],[33,13],[56,64]]])

# TENSOR TENSOR = torch.tensor([[[1,4],[3,1],[5,4]], [[11,14],[33,13],[56,64]]])

In [29]:

# see dimensions  - how many lists are there in the tensor )
print(f"dimension of TENSOR is {TENSOR.ndim}")
# see shape - shape represents how many elements are there in the tensor list ( here we have 3 lists and each list has 4 elements)
print(f"shape of TENSOR is {TENSOR.shape}")
# see item -  Item can be only scalar, thus we went at 0th index of TENSOR , we got 2 lists each, thus we went inside that list and found 3 more lists, then we went inside that list and looked for 0th element and got that item
print(f"item in TENSOR is {TENSOR[0][0][0].item()}")

# see dimensions - how many lists are there in the tensor ) print(f"dimension of TENSOR is {TENSOR.ndim}") # see shape - shape represents how many elements are there in the tensor list ( here we have 3 lists and each list has 4 elements) print(f"shape of TENSOR is {TENSOR.shape}") # see item - Item can be only scalar, thus we went at 0th index of TENSOR , we got 2 lists each, thus we went inside that list and found 3 more lists, then we went inside that list and looked for 0th element and got that item print(f"item in TENSOR is {TENSOR[0][0][0].item()}")

dimension of TENSOR is 3
shape of TENSOR is torch.Size([2, 3, 2])
item in TENSOR is 1

Create tensor based on random numbers¶

In nueral network the way things works is we start by assigning random numbers in tensor and then based on input data, these numbers are corrected.

The flow looks like

Create random tensor ➡️ check it with data ➡️ correct it based on data ➡️ check it with data ➡️ correct it based on data

In [4]:

# Create random TENSOR
random_TENSOR = torch.rand(2, 3) # give size of tensor
random_TENSOR

# Create random TENSOR random_TENSOR = torch.rand(2, 3) # give size of tensor random_TENSOR

Out[4]:

tensor([[0.6117, 0.8783, 0.0312],
        [0.8228, 0.2097, 0.6913]])

In [5]:

# Create random TENSOR
another_random_TENSOR = torch.rand(3,2, 3) # give size of tensor
another_random_TENSOR

# Create random TENSOR another_random_TENSOR = torch.rand(3,2, 3) # give size of tensor another_random_TENSOR

Out[5]:

tensor([[[0.0935, 0.5212, 0.1298],
         [0.9620, 0.9641, 0.4545]],

        [[0.1546, 0.8876, 0.1105],
         [0.9638, 0.4953, 0.2858]],

        [[0.6359, 0.5181, 0.8909],
         [0.8295, 0.2972, 0.3061]]])

In [7]:

# Get all values as zero in tensors
all_zeros = torch.zeros(3, 3)
all_zeros

# Get all values as zero in tensors all_zeros = torch.zeros(3, 3) all_zeros

Out[7]:

tensor([[0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.]])

In [8]:

# Get all values as one in tensors
all_ones = torch.ones(3, 3)
all_ones

# Get all values as one in tensors all_ones = torch.ones(3, 3) all_ones

Out[8]:

tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]])

In [10]:

# Get all values in a range in tensors
all_ranges = torch.range(start=23, end=45, step=3)
all_ranges

# Get all values in a range in tensors all_ranges = torch.range(start=23, end=45, step=3) all_ranges

/var/folders/b5/2422fkd93kq4kkfc_d1vpym00000gn/T/ipykernel_94871/818810665.py:3: UserWarning: torch.range is deprecated and will be removed in a future release because its behavior is inconsistent with Python's range builtin. Instead, use torch.arange, which produces values in [start, end).
  all_ranges = torch.range(start=23, end=45, step=3)

Out[10]:

tensor([23., 26., 29., 32., 35., 38., 41., 44.])

In [11]:

# convert all elements in tensor to zero
ranges_converted_to_zero = torch.zeros_like(all_ranges)
ranges_converted_to_zero

# convert all elements in tensor to zero ranges_converted_to_zero = torch.zeros_like(all_ranges) ranges_converted_to_zero

Out[11]:

tensor([0., 0., 0., 0., 0., 0., 0., 0.])

In [12]:

# convert all elements in tensor to one
ranges_converted_to_one = torch.ones_like(all_ranges)
ranges_converted_to_one

# convert all elements in tensor to one ranges_converted_to_one = torch.ones_like(all_ranges) ranges_converted_to_one

Out[12]:

tensor([1., 1., 1., 1., 1., 1., 1., 1.])