Core Concepts Of Autoencoders Training Ppt

Slide 1

This slide gives an overview of AutoEncoders in Neural Networks. An Autoencoder is a form of Artificial Neural Network used to learn efficient codings of unlabeled input or unsupervised learning. By attempting to regenerate the input from the encoding, the encoding is validated and enhanced.

Slide 2

This slide lists components in an AutoEncoder. These components are an encoder, decoder, and latent space.

Instructor’s Notes: 

• Encoder: Learns to convert incoming data into an encoded representation by compressing (reducing) it
• Decoder: Recovers the original data from the encoded representation as close as possible to the original input
• Latent space: The layer that includes the compressed version of the input data is known as the bottleneck or latent space

Slide 3

This slide depicts types of AutoEncoders. These include undercomplete Autoencoders and regularized Autoencoders which are further divided into sparse Autoencoders and denoising Autoencoders.

Instructor’s Notes: 

• Undercomplete Autoencoders: Undercomplete Autoencoders have a latent space that is smaller than the input dimension. The autoencoder is forced to capture the most important aspects of the training input by learning an incomplete representation
• Regularized Autoencoders: These employ a loss function that promotes the model to have qualities other than the capacity to transfer input to output. Two forms of regularised autoencoders are found in practice: sparse autoencoders and denoising autoencoders
• Sparse Autoencoders: Typically, Sparse Autoencoders are used to learn features for a new job, such as classification. Instead of just operating as an identity function, an Autoencoder that has been regularised to be sparse must respond to unique statistical properties of the dataset it has been trained on
• Denoising Autoencoders: It is no longer necessary to rebuild the input data. Rather than adding a penalty to the loss function, we can change the reconstruction error term of the loss function to obtain an Autoencoder that has the capacity to learn anything meaningful. This can be accomplished by introducing noise to the input image and training the Autoencoder to eliminate it. The encoder will learn a robust representation of the data by extracting the most relevant features

Slide 4

This slide lists applications of AutoEncoders. These include noise removal, dimensionality reduction, anomaly detection, and machine translation.

Instructor’s Notes: 

• Noise Removal: The technique of reducing noise from an image is known as noise removal. Audio and picture noise reduction techniques are available through autoencoders
• Dimensionality Reduction: The transfer of data from a high-dimensional space to a low-dimensional space so that the low-dimensional representation retains some significant aspects of the original data, ideally close to its intrinsic dimension, is known as dimensionality reduction. The encoder section of autoencoders is helpful while performing dimensionality reduction since it learns representations of your input data with considerably reduced dimensionality
• Anomaly Detection: The model is encouraged to learn to exactly recreate the most often seen traits by learning to replicate the most salient features in the training data (under some limitations, of course). When confronted with abnormalities, the model's reconstruction performance should deteriorate. In most circumstances, the autoencoder is trained using only data containing normal instances. The autoencoder will correctly reconstruct "normal" data after training but will fail to do so with unexpected anomalous input
• Machine Translation: Autoencoders have been used in Neural Machine Translation (NMT), a type of machine translation that uses autoencoders. Unlike standard autoencoders, the output is not in the same language as the input. Texts in NMT are viewed as sequences to be encoded into the learning mechanism, whereas sequences in the target language(s) are generated on the decoder side

Slide 5

This slide gives an overview of Variational AutoEncoders in Neural Networks. A Variational Autoencoder (VAE) is a more recent and intriguing approach for autoencoding. VAE presumes that the source data has an underlying probability distribution and then tries to discover the distribution's parameters.

Instructor’s Notes: It's far more challenging to implement a Variational Autoencoder than to implement an Autoencoder. A Variational Autoencoder's principal purpose is to generate new data connected to the original source data.