Elasticsearch Fundamentals
Elasticsearch is a distributed engine that indexes data so you can search, filter, and analyze it quickly. You use it when an application needs search or analytics across data at a scale that benefits from a cluster.
itDatabases and data storage | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Don't Panic
Don't Panic — Elasticsearch Fundamentals
Elasticsearch is a distributed retrieval system for JSON documents. You put documents in, it builds an index, and later it can find the ones that match or summarize the lot. This is more useful than asking a transactional database to become a search engine after lunch, which tends to make lunch unnecessarily exciting.
The first three nouns do most of the work. A document is the JSON record. An index is the named collection holding those records. A mapping tells Elasticsearch what each field means. That last one is the sneaky part: a field is not merely a label on a box. Its mapping determines whether the value can support text search, exact filtering, sorting, or aggregation. The machine is being literal because it has a great many documents to be literal about.
The index is divided into shards, pieces that can live on different nodes. Replica shards copy primary shards so search work and failures do not have to rest on one copy. That explains why Elasticsearch can spread retrieval and analytics across a cluster. It also explains why replicas are not backups. A copy inside the same system is helpful during a failure; a snapshot is what gives you a recovery plan when the system itself has made a poor life choice.
Search and analytics are related but not interchangeable. A search asks which documents match and returns hits, often with relevance scores. A filter applies firm conditions, such as a category or time range. An aggregation asks what the matching documents show: counts, groups, or metrics. If a dashboard needs a number, returning a heroic parade of documents is usually not the point.
One surprise awaits every impatient newcomer. Elasticsearch is near real time. An indexing request can succeed before a subsequent search sees the document. That is normal behavior, not a small rebellion by the data. Build user-facing expectations around it.
Read the intro when you need the architecture and the Elastic Stack context. Use the slides to keep the relationships between documents, mappings, shards, searches, and aggregations straight. Keep the cheatsheet nearby for the operational vocabulary, then use the practice reference and exercise to make the requests real. The quiz is for checking the mental model before cluster sizing, security, and recovery add their usual collection of consequences.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://www.elastic.co/docs/reference/elasticsearch
Supports
- Elasticsearch as a distributed search and analytics engine, scalable data store, and vector database
- Apache Lucene foundation
- Near-real-time indexing, search, storage, and analysis
- Mapping, data management, query languages, aggregations, and API overview
- Elasticsearch reference link rationale
- https://www.elastic.co/docs/get-started/the-stack
Supports
- Elasticsearch as the indexing, querying, and analytics component
- Kibana dashboards, visualizations, and management tools
- Elastic Agent collection and forwarding role
- Logstash ingestion and transformation role
- Nodes, indices, shards, and distributed architecture
- https://www.elastic.co/guide/en/elasticsearch/reference/8.19/getting-started.html
Supports
- Indexed documents as near-real-time searchable
- Search API response fields and hits
- Match query as a standard full-text query
- Index and search tutorial link rationale
- https://www.elastic.co/docs/reference/elasticsearch/rest-apis
Supports
- REST API access to Elasticsearch features
- Document, index, search, and aggregation API categories
- REST API reference link rationale
- https://www.elastic.co/docs/reference/aggregations
Supports
- Aggregations as data analysis and summarization over indexed documents
- Metric, bucket, and pipeline aggregation categories
- Aggregation category quiz answers
- https://www.elastic.co/docs/explore-analyze/query-filter/aggregations
Supports
- Aggregations in search requests
- Grouping and metric analysis use cases
- Aggregations link rationale and analysis quiz answer
- https://www.elastic.co/docs/deploy-manage/production-guidance/elasticsearch-in-production-environments
Supports
- Resilience, scaling, performance, and deployment responsibilities
- Nodes, shards, replicas, and production design
- Snapshot and restore as recovery tools
- Production guidance link rationale
- https://www.elastic.co/docs/deploy-manage/tools
Supports
- Snapshots as point-in-time backups with data, settings, and state
- Recovery planning and snapshot management
- Recovery quiz answer and resilience tools link rationale
- https://www.elastic.co/docs/deploy-manage/security/secure-your-cluster-deployment
Supports
- Security-enabled cluster operation
- Authentication, authorization, and encryption guidance
- https://www.elastic.co/docs/solutions/search/get-started/index-basics
Supports
- Local Docker-based learning setup
- Index, document, mapping, and search exercise commands
- https://www.elastic.co/docs/api/doc/elasticsearch/v8/operation/operation-cluster-health
Supports
- Cluster health status reflects shard allocation
- Green, yellow, and red status interpretation
- Exercise success criterion and Field Notes health signal
- https://www.elastic.co/docs/manage-data/data-store/mapping
Supports
- Explicit and dynamic mappings
- Mapping changes and reindexing implications
- Field Notes mapping-design judgment
- https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-explosion.html
Supports
- Dynamic field growth and mapping explosion symptoms
- Field Notes dynamic-field failure mode
- https://www.elastic.co/docs/deploy-manage/production-guidance/optimize-performance/size-shards
Supports
- Representative workload benchmarking for shard strategy
- Shard-size and recovery tradeoffs
- Field Notes shard-sizing judgment
- https://discuss.elastic.co/t/elasticsearch-1-0-0-is-now-ga/15766
Supports
- Elasticsearch 1.0 general availability in February 2014
- https://www.elastic.co/blog/elasticsearch-2-0-0-released
Supports
- Elasticsearch 2.0 general availability in October 2015
- Query and filter unification and pipeline aggregations
- https://www.elastic.co/blog/elasticsearch-5-0-0-released
Supports
- Elasticsearch 5.0 general availability in October 2016
- Ingest-node pipelines
- https://www.elastic.co/blog/elasticsearch-6-0-0-released
Supports
- Elasticsearch 6.0 general availability in November 2017
- Sequence IDs and operations-based recovery
- https://www.elastic.co/blog/elasticsearch-7-0-0-released
Supports
- Elasticsearch 7.0 general availability in April 2019
- New cluster coordination and removal of minimum_master_nodes
- https://www.elastic.co/blog/whats-new-elastic-7-10-0-searchable-snapshots-lens-user-experience-monitoring/
Supports
- Elastic 7.10 release in November 2020
- Searchable snapshots and data-tier evolution
- https://www.elastic.co/about/press/elastic-announces-the-general-availability-of-elastic-8-0
Supports
- Elastic 8.0 general availability in February 2022
- Security-by-default setup with generated tokens and certificates
- https://www.elastic.co/kr/blog/whats-new-elasticsearch-8-8-0
Supports
- Elasticsearch 8.8 general availability in May 2023
- Vector search and hybrid-ranking enhancements
- https://opensearch.org/
Supports
- OpenSearch landscape entry
- https://solr.apache.org/
Supports
- Apache Solr landscape entry
- https://www.algolia.com/
Supports
- Algolia landscape entry
- https://typesense.org/
Supports
- Typesense landscape entry
- https://www.meilisearch.com/
Supports
- Meilisearch landscape entry
