Understanding Gradient Descent Training Ppt

Slide 1

This slide introduces the concept of gradient descent. Gradient Descent is an optimization process used in Machine Learning algorithms to minimize the cost function (the error between the actual and predicted output). It is mainly used to update the learning model's parameters.

Slide 2

This slide lists types of gradient descent. These include batch gradient descent, stochastic gradient descent, and mini-batch gradient descent.

Instructor’s Notes: 

• Batch Gradient Descent: Batch gradient descent adds the errors for each point in a training set before updating the model after all training instances have been reviewed. This process is known as the Training Epoch. Batch gradient descent usually gives a steady error gradient and convergence, although choosing the local minimum rather than the global minimum isn't always the best solution
• Stochastic Gradient Descent: Stochastic gradient descent creates a training epoch for each example in the dataset and changes the parameters of each training example, sequentially. These frequent updates can provide greater detail and speed, but they can also produce noisy gradients, which can aid in surpassing the local minimum and locating the global one
• Mini-Batch Gradient Descent: Mini-batch gradient descent combines the principles of both batch gradient descent with stochastic gradient descent. It divides the training dataset into distinct groups and updates them separately. This method balances batch gradient descent's computing efficiency and stochastic gradient descent's speed