ARTIFICIAL NEURAL NETWORKS & DEEP LEARNING

ARTIFICIAL NEURAL NETWORKS (ANN)

Introduction to Artificial Neural Networks

Artificial Neural Networks are machine learning models inspired by the human brain.

The human brain contains billions of neurons connected together.
Similarly, ANN consists of artificial neurons connected in layers to process information.

Each neuron receives input, processes it, and produces output.

Basic Structure of ANN

• Input Layer – receives input data
• Hidden Layer(s) – performs computation
• Output Layer – produces final result

ANN learns by adjusting weights between neurons during training.

Applications

• Image recognition
• Speech recognition
• Natural language processing
• Medical diagnosis
• Fraud detection

Perceptron

The Perceptron is the simplest type of neural network proposed by Frank Rosenblatt in 1957.

It is used for binary classification problems.

Example:

• Spam vs Not Spam
• Yes vs No

Working of Perceptron

1. Input values are multiplied by weights.
2. The weighted sum is calculated.
3. A threshold function determines the output.

Mathematical Representation

$y=f({\sum }_{i=1}^n{w}_i{x}_i+b)$

Where:

Symbol	Meaning
xᵢ	Input value
wᵢ	Weight
b	Bias
f	Activation function
y	Output

Limitations

• Can only classify linearly separable data
• Cannot solve complex problems like XOR

Multilayer Perceptron (MLP)

Multilayer Perceptron is an advanced neural network that contains multiple layers.

Structure:

Input Layer
↓
Hidden Layer(s)
↓
Output Layer

The hidden layers help the network learn complex patterns.

Features of MLP

Feature	Explanation
Multiple Layers	Allows complex learning
Non-linear Activation	Handles nonlinear problems
Supervised Learning	Requires labeled data

Applications

• Image classification
• Speech recognition
• Pattern recognition

Gradient Descent

Gradient Descent is an optimization algorithm used to minimize error in neural networks.

The goal is to adjust weights so that prediction error becomes minimal.

Idea

The algorithm moves step-by-step toward the minimum error point.

Weight update rule:

$w_{new} = w_{old} - \eta \frac{\partial E}{\partial w}$

Where:

Symbol	Meaning
η (eta)	Learning rate
E	Error
w	Weight

If error decreases → learning improves.

Delta Rule

The Delta Rule is a learning rule used to update weights in a perceptron or neural network.

It adjusts weights according to the difference between actual output and predicted output.

Formula

$\Delta w_i = \eta (t - y)x_i$

Where:

Symbol	Meaning
t	Target output
y	Predicted output
xᵢ	Input
η	Learning rate

Meaning: If prediction is wrong → weights are adjusted to reduce error.

Multilayer Networks

Multilayer networks contain more than one hidden layer.

These networks can learn very complex relationships in data.

Layers in Multilayer Network

Layer	Function
Input Layer	Receives data
Hidden Layer	Processes data
Output Layer	Produces final prediction

Advantages

• Can solve nonlinear problems
• High prediction accuracy
• Flexible architecture

Example Applications

• Face recognition
• Voice assistants
• Autonomous vehicles

Backpropagation Algorithm

Backpropagation is the most important training algorithm used in neural networks. It helps update weights by propagating the error backward through the network.

Steps of Backpropagation

1. Initialize weights randomly.
2. Perform forward propagation.
3. Calculate error between predicted and actual output.
4. Propagate error backward through network.
5. Update weights using gradient descent.
6. Repeat until error becomes minimal.

Error Function

$E = \frac{1}{2}\sum (t - y)^2$

Where:

Symbol	Meaning
t	Target output
y	Predicted output

Importance: Backpropagation allows neural networks to learn complex patterns efficiently.

Generalization

Generalization means the ability of a neural network to perform well on new unseen data. A good neural network should not only memorize training data but also predict correctly for new data.

Example

Training data → Students’ marks
Model predicts → future students' results

Problems Affecting Generalization

Problem	Meaning
Overfitting	Model learns training data too well
Underfitting	Model too simple

Solutions

• Regularization
• Dropout
• Cross-validation

Unsupervised Learning in Neural Networks

In unsupervised learning, the network learns patterns from data without labeled outputs. The model identifies hidden structures in data.

Example: Customer segmentation.

Self-Organizing Map (SOM) Algorithm

Self-Organizing Map is an unsupervised neural network developed by Teuvo Kohonen. It is used for data visualization and clustering.

Characteristics of SOM

Feature	Explanation
Unsupervised Learning	No labeled data required
Competitive Learning	Neurons compete to represent input
Topological Mapping	Similar data grouped together

Steps of SOM Algorithm

1. Initialize weights randomly.
2. Present input vector to network.
3. Find Best Matching Unit (BMU).
4. Update weights of BMU and its neighbors.
5. Repeat until stable mapping is achieved.

Applications

• Data visualization
• Image compression
• Pattern recognition
• Market segmentation

Variants of SOM

Some improved versions of SOM include:

Variant	Description
Growing SOM	Network expands automatically
Hierarchical SOM	Multiple layers of maps
Adaptive SOM	Adjusts learning parameters

Summary Table

Topic	Key Idea
Perceptron	Single-layer neural network
Multilayer Perceptron	Neural network with hidden layers
Gradient Descent	Optimization method to minimize error
Delta Rule	Updates weights based on error
Backpropagation	Training algorithm for neural networks
Generalization	Ability to predict new data
SOM	Unsupervised neural network for clustering

Important MCA Exam Questions

1. Explain Artificial Neural Networks with diagram.
2. What is Perceptron? Explain its working.
3. Explain Multilayer Perceptron (MLP).
4. Derive the Backpropagation algorithm.
5. Explain Gradient Descent and Delta Rule.
6. Explain Self-Organizing Map (SOM) algorithm. 

SOM ALGORITHM AND ITS VARIANTS

Self-Organizing Map (SOM) Algorithm

Self-Organizing Map (SOM) is an unsupervised neural network algorithm developed by Teuvo Kohonen. It is used to cluster data and visualize high-dimensional data in a lower dimension (usually 2D).

The main idea of SOM is that similar data points are mapped close together on a grid.

Features of SOM

Feature	Explanation
Unsupervised Learning	No labeled output required
Competitive Learning	Neurons compete to represent input
Topology Preservation	Similar data points remain close

Steps of SOM Algorithm

1. Initialize network weights randomly.
2. Present an input vector to the network.
3. Calculate distance between input vector and neuron weights.
4. Identify the Best Matching Unit (BMU).
5. Update weights of BMU and its neighbors.
6. Repeat process for all inputs until network stabilizes.

Applications

• Data clustering
• Image processing
• Market segmentation
• Data visualization

Variants of SOM

Several improved versions of SOM exist.

Variant	Description
Growing SOM	Network size increases automatically when new patterns appear
Hierarchical SOM	Uses multiple layers of maps for complex data
Adaptive SOM	Adjusts learning rate automatically
Dynamic SOM	Map structure changes during training

These variants improve scalability and learning efficiency.

DEEP LEARNING

Introduction to Deep Learning

Deep Learning is a subset of Machine Learning that uses neural networks with many layers (deep neural networks). These deep networks can learn complex patterns automatically from large datasets.

Difference between ML and Deep Learning

Feature	Machine Learning	Deep Learning
Data Requirement	Moderate	Very large
Feature Extraction	Manual	Automatic
Model Complexity	Simple models	Deep neural networks

Applications

• Face recognition
• Speech recognition
• Autonomous vehicles
• Medical image analysis

CONVOLUTIONAL NEURAL NETWORK (CNN)

Convolutional Neural Networks are deep learning models mainly used for image and video processing.

CNN automatically extracts important features from images.

Examples:

• Detecting objects in images
• Face recognition
• Medical image analysis

Basic CNN Architecture

Input Image
↓
Convolution Layer
↓
Activation Function
↓
Pooling Layer
↓
Fully Connected Layer
↓
Output (classification)

TYPES OF LAYERS IN CNN

CNN contains several types of layers.

1. Convolutional Layer

This layer performs convolution operation on input images to extract features.

Features may include:

• Edges
• Shapes
• Textures

Each convolution layer uses filters (kernels) to scan the image.

Example: Filter detects edges or patterns in image.

Purpose:

• Feature extraction

2. Activation Function

• Activation functions introduce non-linearity in neural networks.
• Without activation functions, neural networks behave like simple linear models.

Common activation functions:

Function	Description
ReLU	Most widely used
Sigmoid	Used for probabilities
Tanh	Output between -1 and 1

Example: ReLU function

f(x) = max(0, x)

Advantages:

• Faster training
• Avoids vanishing gradient problem

3. Pooling Layer

Pooling layer reduces the size of feature maps.

This helps:

• Reduce computation
• Avoid overfitting

Types of pooling:

Type	Description
Max Pooling	Takes maximum value
Average Pooling	Takes average value

Example: 2×2 pooling reduces image dimension.

4. Fully Connected Layer

This is the final layer of CNN.

It connects all neurons from previous layer to produce final classification result.

Example outputs:

• Cat
• Dog
• Human face

Purpose:

• Decision making

5. CONCEPT OF CONVOLUTION

Convolution is a mathematical operation used to extract features from input data.

In CNN, convolution applies filters to detect patterns.

1D Convolution

Used for sequential data.

Example:

• Audio signals
• Time-series data
• Text processing

Example application:
Speech recognition.

2D Convolution

Used mainly for image processing.

The filter slides across image pixels to detect patterns like:

• Edges
• Shapes
• Textures

Example application:
Object detection in images.

TRAINING OF CNN NETWORK

Training a CNN involves learning the optimal weights and filters.

Steps of Training

1. Initialize network parameters.
2. Forward propagate input through layers.
3. Calculate prediction error.
4. Use backpropagation algorithm.
5. Update weights using gradient descent.
6. Repeat process until error becomes minimal.

CASE STUDY – CNN FOR DIABETIC RETINOPATHY

Diabetic Retinopathy is an eye disease caused by diabetes.

CNN models can analyze retinal images to detect early signs of the disease.

Process

1. Collect retinal images dataset.
2. Preprocess images.
3. Train CNN model.
4. Model identifies patterns indicating disease.
5. Predict severity of retinopathy.

Benefits

• Early detection
• Faster diagnosis
• Supports doctors in medical decisions

Companies like Google Health have developed such AI systems.

BUILDING A SMART SPEAKER

Smart speakers like Amazon Alexa or Google Home use deep learning.

Components

1. Speech Recognition Model
   Converts voice into text.
2. Natural Language Processing (NLP)
   Understands user commands.
3. Intent Recognition
   Determines what the user wants.
4. Response Generation

Example: User: “Play music”

System:

• Recognizes voice
• Understands command
• Plays music.

SELF-DRIVING CAR

Self-driving cars use deep learning and computer vision.

Sensors used:

• Cameras
• Radar
• LIDAR

Role of Deep Learning

Task	Function
Object Detection	Detect vehicles and pedestrians
Lane Detection	Identify road lanes
Traffic Sign Recognition	Detect traffic signals
Path Planning	Determine driving route

Companies using this technology:

• Tesla
• Google Waymo
• Uber

Summary Table

Topic	Key Idea
SOM	Unsupervised clustering algorithm
Deep Learning	Neural networks with many layers
CNN	Deep learning model for images
Convolution	Feature extraction operation
Pooling	Reduces data size
Activation Function	Introduces non-linearity
CNN Training	Backpropagation + gradient descent
Applications	Medical AI, smart speakers, self-driving cars

Important MCA Exam Questions

1. Explain Deep Learning with examples.
2. Explain CNN architecture and its layers.
3. What is Convolution (1D and 2D)?
4. Explain Self-Organizing Map algorithm.
5. Write applications of Deep Learning in real world.
6. Explain CNN for diabetic retinopathy detection