Natural Language Processing
Natural language processing is a branch of artificial intelligence that enables computers to read, interpret, and generate human language in text or speech form. It powers search engines, translation services, chatbots, voice assistants, and text analytics tools.
itArtificial intelligence and machine learning | OpenSkills.info
Intro
Natural Language Processing
Natural language processing is the set of methods that let machines work with human language — reading text, understanding speech, generating responses, and extracting structured information from unstructured words. The field sits at the intersection of linguistics, computer science, and machine learning.
You encounter NLP every time you use a search engine, ask a voice assistant a question, read a machine-translated article, or receive an auto-generated email summary. Behind each of these is a system that converts raw language into representations a computer can operate on, then maps those representations to useful outputs.
Why NLP exists
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://web.stanford.edu/class/cs224n/index.html
Supports
- NLP enables machines to read, interpret, and generate human language in text or speech form
- https://arxiv.org/abs/1706.03762
Supports
- The Transformer architecture uses self-attention to process all positions in parallel, introduced in 2017
- https://arxiv.org/abs/1810.04805
Supports
- BERT uses masked language modeling to learn bidirectional contextual representations
- https://web.stanford.edu/class/cs224n/index.html
Supports
- GPT uses causal (left-to-right) language modeling to predict the next token from preceding context
- https://web.stanford.edu/class/cs224n/index.html
Supports
- Word2Vec and GloVe learn static dense vector representations where semantically similar words are near each other
- https://huggingface.co/learn/nlp-course
Supports
- BPE tokenization iteratively merges frequent byte pairs to create a subword vocabulary
- https://web.stanford.edu/class/cs224n/index.html
Supports
- BLEU measures translation quality based on n-gram precision with a brevity penalty
- https://arxiv.org/abs/1706.03762
Supports
- Self-attention computes scores as dot product of query and key vectors divided by square root of dimension
- https://arxiv.org/abs/2005.11401
Supports
- Retrieval-augmented generation combines a retriever with a generator to ground outputs in source documents
- https://web.stanford.edu/class/cs224n/index.html
Supports
- ELMo introduced context-dependent word representations using bidirectional LSTMs
- https://arxiv.org/abs/1706.03762
Supports
- Attention in seq2seq models lets the decoder focus on relevant encoder positions at each generation step
- https://arxiv.org/abs/1810.04805
Supports
- Fine-tuning adapts a pretrained model to a downstream task using task-specific labeled data
- https://web.stanford.edu/class/cs224n/index.html
Supports
- Language models trained on human text can reproduce and amplify biases present in training data
- https://arxiv.org/abs/1706.03762
Supports
- Attention cost grows quadratically with sequence length in standard Transformer implementations