Blockchain Indexing and Subgraphs
Blockchain indexing turns raw chain activity into queryable data. Subgraphs define which contracts, events, and entities to index, and The Graph Network turns those definitions into GraphQL APIs applications can query without running their own nodes.
itDistributed systems, messaging, and integration | OpenSkills.info
Recommended first:web3-application-integration
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Don't Panic
Don't Panic: Blockchain Indexing and Subgraphs
A blockchain remembers transactions. It is considerably less helpful when an application asks for a tidy list of everything a particular contract did last week. The chain exposes blocks, logs, and state through RPC. It does not hand you application records with filters, joins, and convenient names. You can collect the raw data yourself, but then you have built an indexer, whether or not you meant to.
A subgraph is a recipe for that indexer. Its manifest names the chain, contracts, and events to watch. Its schema names the records an application wants to query. Its mappings turn each matching event into one or more stored records. Graph Node follows the recipe and serves the result through GraphQL. That is the whole path: watch, transform, store, query. The files are small; the historical chain they may have to process is not.
The first surprise is that the schema is an operational choice. If a token has many transfers, copying every transfer ID into one growing token array seems natural. Each new event then rewrites a larger array. Store the token reference on each transfer instead, and let @derivedFrom expose the collection from the token. The relationship is still queryable without asking one entity to carry the entire history.
The second surprise is that a successful GraphQL response can be old news. A subgraph must first catch up with the chain. Its _meta field reports the latest block it has indexed and whether indexing errors exist. Compare that block with the chain head when freshness matters. A dashboard that loads is not necessarily a dashboard that has caught up.
Subgraph Studio is the workbench. A staged deployment is for building, testing, and checking logs. Publishing is a separate onchain step that makes the subgraph available on The Graph Network, where Indexers serve queries. An application can still use direct RPC for a current contract value or to submit a transaction. The subgraph supplies the historical and cross-contract view that raw RPC does not arrange for you.
If the ordinary mapping path is too slow for a large backfill or a streaming transformation, Substreams offers parallel processing over block streams. It can send results to a database or feed a subgraph when GraphQL is still the right public interface. The distinction is practical: decide what data the application must ask for, then choose the processing and query path that can deliver it.
Read the Intro for the full data path and network roles. Use the Cheatsheet when choosing handler types, entity relationships, and query pagination. The Practice Reference and Exercise turn those decisions into a staged subgraph you can inspect before anyone depends on it.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://thegraph.com/docs/en/about/
Supports
- Why blockchain applications use indexed data and how Subgraphs and Substreams fit
- Start here: the platform's own statement of the four reasons blockchains resist querying and the product split between Subgraphs and Substreams this course builds on.
- Grounds quiz item blockchain-indexing-and-subgraphs-01
- Grounds quiz item blockchain-indexing-and-subgraphs-14
- Grounds quiz item blockchain-indexing-and-subgraphs-15
- https://thegraph.com/docs/en/subgraphs/overview/
Supports
- Subgraph definition, development model, and network roles
- The definition, the three definition files, the subgraph lifecycle, and the network roles (Indexer, Curator, Delegator) as The Graph documents them.
- Grounds quiz item blockchain-indexing-and-subgraphs-02
- Grounds quiz item blockchain-indexing-and-subgraphs-11
- https://thegraph.com/docs/en/subgraphs/quick-start/
Supports
- Graph CLI steps, Studio staging, network publication, and query plan
- The complete build path from graph init through Studio deployment to publishing, including the exact commands the practice reference uses.
- Grounds quiz item blockchain-indexing-and-subgraphs-04
- Grounds quiz item blockchain-indexing-and-subgraphs-10
- https://thegraph.com/docs/en/subgraphs/developing/creating/subgraph-manifest/
Supports
- Manifest fields, event and call handlers, templates, start blocks, and pruning
- Full field-by-field manifest documentation: data sources, handler types, data source templates, start blocks, indexerHints.prune, and specVersion features.
- Grounds quiz item blockchain-indexing-and-subgraphs-05
- Grounds quiz item blockchain-indexing-and-subgraphs-06
- Grounds quiz item blockchain-indexing-and-subgraphs-09
- https://thegraph.com/docs/en/subgraphs/developing/creating/ql-schema/
Supports
- Entities, Bytes IDs, immutable records, relationships, and derived fields
- Entity rules (id types, immutable vs mutable), relationship storage, @derivedFrom reverse lookups, and the mapping-table pattern for many-to-many links.
- Grounds quiz item blockchain-indexing-and-subgraphs-07
- https://thegraph.com/docs/en/subgraphs/developing/creating/assemblyscript-mappings/
Supports
- Mapping handlers and entity writes
- Handler syntax, recommended id construction, and the codegen cycle between schema edits and mapping types.
- Grounds quiz item blockchain-indexing-and-subgraphs-03
- https://thegraph.com/docs/en/subgraphs/querying/graphql-api/
Supports
- GraphQL filtering, pagination, sorting, metadata, and historical queries
- The complete query surface: filtering operators, sorting, pagination limits, time-travel queries, fulltext search, and the _meta object.
- Grounds quiz item blockchain-indexing-and-subgraphs-08
- https://thegraph.com/docs/en/substreams/overview/
Supports
- Parallel block processing and sinks
- The parallel indexing alternative: Rust modules, WASM execution, sinks, and the non-EVM chains and trace-level data it covers.
- Grounds quiz item blockchain-indexing-and-subgraphs-14
- https://thegraph.com/docs/en/indexing/overview/
Supports
- Indexer roles, stake, and data service operations
- The indexer-side stack (Graph Node, PostgreSQL, IPFS, indexer agents and services), staking and rewards mechanics, and the hardware envelopes the network runs on.
- Grounds quiz item blockchain-indexing-and-subgraphs-11
- Grounds quiz item blockchain-indexing-and-subgraphs-12
- Grounds quiz item blockchain-indexing-and-subgraphs-13
- https://thegraph.com/blog/
Supports
- Official project announcements for course update tracking
- https://github.com/sindresorhus/awesome
Supports
- Discovery index for topic-specific awesome lists
- https://github.com/balakhonoff/awesome-subgraphs
Supports
- Curated subgraph developer projects used for Awesome Links selection
- https://thegraph.com/docs/en/subgraphs/querying/best-practices/
Supports
- Static queries, asking only for needed fields, single-request multi-record patterns, and fragment usage for application integrations.
- https://thegraph.com/docs/en/graph-horizon/overview/
Supports
- The protocol upgrade turning staking and payments into reusable primitives with SubgraphService as the first data service, with links to the underlying GIPs.
- https://github.com/graphprotocol/graph-improvement-proposals/blob/main/gips/0066-graph-horizon.md
Supports
- Primary specification for the data services protocol: provisions, slashable delegation, the payments protocol, and the Data Service framework contract.
- Grounds quiz item blockchain-indexing-and-subgraphs-13
- https://github.com/graphprotocol/graph-node
Supports
- The implementation behind local indexing: Docker setup for development, source build prerequisites, and the environment-variable configuration surface.
- Product site or repository for Graph Node and its role in indexing
- https://github.com/graphprotocol/graph-tooling/tree/main/packages/cli
Supports
- The tooling behind graph init, codegen, build, deploy, and publish, plus the TypeScript library mappings import from @graphprotocol/graph-ts.
- https://github.com/messari/subgraphs
Supports
- Study concrete DeFi entity schemas and mappings after learning the manifest-schema-mapping pipeline. Its shared schemas show how a team makes data comparable across related protocols.
- https://github.com/protofire/subgraph-toolkit
Supports
- Explore helper code for developing subgraphs once the basic Graph CLI workflow is familiar. The repository documents the utilities it offers.
- https://github.com/0xPlaygrounds/subgrounds
Supports
- Use a Python client to inspect and transform data from existing subgraphs, including larger paginated collections.
- https://thegraph.com/docs/en/subgraphs/developing/developer-faq/
Supports
- Grounds quiz item blockchain-indexing-and-subgraphs-05
- https://ethereum.org/developers/docs/apis/json-rpc/
Supports
- Grounds quiz item blockchain-indexing-and-subgraphs-15
- https://thegraph.com/blog/announcing-the-graphs-grt-sale/
Supports
- Documents the milestone: The Graph is introduced
- Documents the milestone: Hosted service launches
- Documents the milestone: Mission Control testnet begins
- https://thegraph.com/blog/the-graph-network-launches-mainnet/
Supports
- Documents the milestone: The Graph Network mainnet launches
- https://thegraph.com/blog/curation-live/
Supports
- Documents the milestone: Graph Explorer and Subgraph Studio launch
- https://thegraph.com/blog/transitioning-to-decentralized-graph-network/
Supports
- Documents the milestone: Hosted service migration is announced
- https://thegraph.com/blog/mips-multi-chain-indexing-incentivized-program/
Supports
- Documents the milestone: Multi-Chain Incentivized Program begins
- https://thegraph.com/blog/substreams-powered-subgraphs/
Supports
- Documents the milestone: Substreams-powered subgraphs reach the network
- https://thegraph.com/blog/sunray-complete/
Supports
- Documents the milestone: Sunray phase completes
- https://thegraph.com/blog/sunsetting-hosted-service/
Supports
- Documents the milestone: Hosted service stops serving queries
- Supports Field Notes card: Hosted service URLs are a migration clue
- https://thegraph.com/
Supports
- Product site or repository for The Graph and its role in indexing
- https://substreams.dev/
Supports
- Product site or repository for Substreams and its role in indexing
- https://goldsky.com/products/subgraphs
Supports
- Product site or repository for Goldsky Subgraphs and its role in indexing
- https://www.sqd.dev/
Supports
- Product site or repository for Subsquid and its role in indexing
- https://envio.dev/
Supports
- Product site or repository for Envio HyperIndex and its role in indexing
- https://thegraph.com/blog/improve-subgraph-performance-reduce-eth-calls/
Supports
- Supports Field Notes card: Backfill cost hides in each handler
- https://thegraph.com/blog/improve-subgraph-performance-avoiding-large-arrays/
Supports
- Supports Field Notes card: An entity array can become the bottleneck
