The Building Blocks of Neural Networks

Neurons?

At the core of deep learning lie artificial neural networks, computational models inspired by the architecture and functioning of the human brain. These networks consist of layers of interconnected nodes, or "neurons," each performing simple computations. Neurons in the human brain have many 'input' connections called dendrites and usually one larger output neuron called the axon. Similarly, Artificial computational neurons have inputs which are summed and processed via mathematical activation function before being sent as a signal to the next neuron.

Neural Network?

A typical neural network has a more complex structure and multiple types of layers. Neurons are located mostly in the part of the neural network we call the hidden layer. But what is the hidden layer? To understand that lets explore the architecture of Deep Neural Networks...

1. Input Layer: Receives raw data such as numbers, images, text, or audio, and forwards it to the rest of the network for processing.

2. Hidden Layers: Multiple layers that perform successive transformations on the input data, extracting and refining features at various levels of abstraction. They are called hidden layers because the human users usually interact with either the information which is input or the output of the Neural Network, but not much with the neurons that are between them. That's why this layer is considered as 'hidden' and it is the most important layer in Neural Networks. For a Neural Network to be 'Deep' it should have at least 2 hidden layers. In reality Deep Learning models can have hundreds or more hidden layers. These layers lay the foundation of Deep Learning and enable it to find hidden patterns in the data and learn effectively from large datasets.

3. Output Layer: Generates the final prediction or classification based on the processed information.

Each neuron processes inputs by applying weights and biases, followed by an activation function like ReLU (Rectified Linear Unit) or sigmoid. This structure enables neural networks to learn non-linear relationships and represent complex functions, making them exceptionally powerful for a variety of tasks

The activation functions control the way how the neurons communicate with each other which is very important to enable the learning process for ANNs. They usually set specific thresholds, as rules weather the neuron should 'fire' signal towards the next layer or not. These are some of the principles that lay the foundation of how Deep Learning models are built, but how do they actually learn?

Lets explore how do machines learn!