Deep Learning Fundamentals
Deep learning trains neural networks with many layers to learn hierarchical representations from data. It powers image recognition, language understanding, speech synthesis, and generative models by automatically discovering the features that traditional machine learning requires engineers to design by hand.
itArtificial intelligence and machine learning | OpenSkills.info
Intro
Deep Learning Fundamentals
Deep learning trains layered neural networks to learn useful representations from data. Instead of asking you to hand-design every pattern, a model learns parameters that transform input into an output.
That sentence contains the whole idea. The rest of the field asks four practical questions:
- What representation should the network learn?
- How should the network turn that representation into a prediction?
- How should training change the parameters?
- How do you know the result works beyond the training data?
Where deep learning fits
Deep learning is part of machine learning. Machine learning builds models from data. Deep learning focuses on neural networks with multiple layers of learned transformations.
A conventional program follows rules written by a developer. A trained neural network follows a function shaped by examples and an optimization process. You still write code around the model. You also choose the data, objective, architecture, evaluation, and operating constraints.
Deep learning is useful when the input contains complex patterns that are difficult to specify as explicit rules. Images, audio, text, and other high-dimensional data are common examples. It also supports structured-data tasks when enough relevant data and a suitable objective are available.
It is not the default answer to every prediction problem. A simpler model can be easier to train, inspect, and operate. Clear business rules should remain rules when no learned inference is needed.
The forward path
A neural network receives input as a tensor, which is a multidimensional array. Each layer applies a transformation. Learnable weights and biases control that transformation.
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://www.deeplearningbook.org/
Supports
- Deep learning as representation learning with multilayer neural networks
- Machine learning foundations, feedforward networks, regularization, optimization, convolutional networks, sequence modeling, and practical methodology
- Parameters, loss functions, gradient-based optimization, generalization, overfitting, and data-split roles
- Limits of training objectives and the need to compare model choices through evaluation
- https://developers.google.com/machine-learning/crash-course/neural-networks
Supports
- Neural networks as architectures for learning nonlinear patterns
- Nodes, hidden layers, activation functions, inference, backpropagation, and multiclass output
- Nonlinear activations as necessary for representing nonlinear relationships with stacked layers
- https://developers.google.com/machine-learning/crash-course/neural-networks/backpropagation
Supports
- Backpropagation as the training process that computes parameter gradients
- Gradient descent and learning rate as parts of iterative parameter updates
- The relationship among forward passes, loss, backward passes, and optimization
- https://developers.google.com/machine-learning/crash-course/overfitting/regularization
Supports
- Generalization and overfitting diagnosed with held-out data and loss curves
- Model complexity and L2 regularization as considerations in controlling overfitting
- Separation of training, validation, and test roles
- https://developers.google.com/machine-learning/crash-course/classification/accuracy-precision-recall
Supports
- Accuracy limitations for imbalanced data
- Precision and recall as class-error measures
- Metric selection based on the consequences of false positives and false negatives
- https://developers.google.com/machine-learning/crash-course/production-ml-systems
Supports
- A model as one component of a production machine-learning system
- Deployment testing, pipeline monitoring, training and inference concerns
- Operational requirements beyond an offline model metric
- https://docs.pytorch.org/tutorials/beginner/basics/intro.html
Supports
- A complete workflow covering tensors, datasets, models, automatic differentiation, optimization, and model persistence
- Tensors as the data structure for model input, output, and parameters
- Training as repeated data, model, loss, gradient, and update operations
- https://docs.pytorch.org/tutorials/beginner/basics/autogradqs_tutorial.html
Supports
- Automatic differentiation over a computation graph
- Backpropagation of loss gradients to parameters with the chain rule
- Gradient tracking for training and its disabling during inference
- https://docs.pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial
Supports
- Neural-network modules, learnable parameters, forward computation, loss functions, backpropagation, and parameter updates
- Mini-batch tensor input and gradient accumulation behavior
- https://doi.org/10.1038/nature14539
Supports
- Representation learning and multilayer neural-network foundations
- Convolutional and recurrent architectures
- Applications in speech recognition, visual recognition, and other domains
- https://arxiv.org/abs/1706.03762
Supports
- The original Transformer architecture based on attention
- Replacement of recurrence and convolution with attention in the paper's sequence-transduction design
- Parallelization benefits of that architecture during training