Neural Networks
Neural networks are computing systems inspired by biological brains, built from layers of interconnected nodes that learn patterns from data. They power image recognition, language translation, recommendation engines, and many other tasks where traditional programming rules would be impractical to write by hand.
itArtificial intelligence and machine learning | OpenSkills.info
Intro
Neural Networks
A neural network is a computational model that learns to map inputs to outputs by adjusting internal parameters. You feed it examples, and it discovers the patterns that connect those examples to correct answers. The result is a system that generalizes — it handles inputs it has never seen before.
Neural networks matter because they solve problems where writing explicit rules is impossible or impractical. Recognizing a face in a photo, translating a sentence between languages, predicting equipment failure from sensor readings — these tasks require the system to learn structure from data rather than follow human-authored instructions.
Why neural networks exist
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://developers.google.com/machine-learning/crash-course/neural-networks
Supports
- Neural networks learn to map inputs to outputs by adjusting internal parameters through training on examples
- https://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html
Supports
- A neuron computes a weighted sum of inputs, adds a bias, and applies a nonlinear activation function
- https://developers.google.com/machine-learning/crash-course/neural-networks
Supports
- Without nonlinear activation functions, multiple layers collapse to a single linear transformation
- https://developers.google.com/machine-learning/crash-course/neural-networks/backpropagation
Supports
- Backpropagation computes gradients of the loss with respect to each weight using the chain rule
- https://developers.google.com/machine-learning/crash-course/neural-networks
Supports
- ReLU is the default activation for hidden layers because it avoids the vanishing gradient problem
- https://pytorch.org/tutorials/beginner/basics/buildmodel_tutorial.html
Supports
- CNNs use learned convolutional filters that slide across spatial dimensions to detect local patterns
- https://arxiv.org/abs/1512.03385
Supports
- ResNet introduced skip connections enabling training of networks with 100+ layers (2015)
- https://arxiv.org/abs/1706.03762
Supports
- The Transformer architecture uses self-attention to process sequences in parallel, introduced in 2017
- https://ocw.mit.edu/courses/6-7960-deep-learning-fall-2024/
Supports
- LSTM and GRU add gating mechanisms to address vanishing gradients in recurrent networks
- https://pytorch.org/tutorials/beginner/basics/buildmodel_tutorial.html
Supports
- Transfer learning reuses pretrained models to reduce data and compute requirements for specific tasks
- https://developers.google.com/machine-learning/crash-course/neural-networks
Supports
- Adam optimizer combines momentum and adaptive per-parameter learning rates
- https://developers.google.com/machine-learning/crash-course/neural-networks
Supports
- Dropout randomly zeroes neuron activations during training to prevent overfitting
- https://ocw.mit.edu/courses/6-7960-deep-learning-fall-2024/
Supports
- Batch normalization normalizes layer inputs, reducing internal covariate shift and enabling higher learning rates
- https://ocw.mit.edu/courses/6-7960-deep-learning-fall-2024/
Supports
- AlexNet (2012) demonstrated deep CNNs trained on GPUs could win ImageNet classification
- https://ocw.mit.edu/courses/6-7960-deep-learning-fall-2024/
Supports
- Vision Transformer (ViT, 2020) applies the Transformer architecture to image patches