API Testing
API testing checks a software interface by sending it requests directly and verifying the responses, status codes, data, and side effects, with no user interface involved. It catches broken behavior, bad error handling, and authorization gaps in the services that applications and other systems call over the network.
itSoftware engineering | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Don't Panic
Don't Panic: API Testing
API testing means talking to a piece of software the way other software talks to it: through its application programming interface, the programmatic front door, with no screen in the way. A test sends a request, reads the reply, and decides whether the reply keeps the promises the interface made. It is less dramatic than clicking through an app, and considerably more honest, because the API has nowhere to hide a bad answer behind a nice layout.
The alternative is to check everything through the user interface, with a browser test clicking buttons for every rule. That works, slowly, and with more ways to fail for reasons unrelated to the rule. API tests sit one floor down, where the data actually changes hands: faster and more stable than browser tests, while still exercising the real service rather than a model of it.
Three ideas carry most of the weight. First, a reply has layers. The status code is the three-digit verdict, so 201 means created, 401 means "who are you?", 403 means "I know who you are, and no", and anything in the five hundreds means the server fell over on a perfectly good request. Headers and the body come next. Checking that a status is merely "some kind of success" is how a 200 sneaks past where a 201 was promised.
Second, a reply is only a claim. An update that says "done" should be followed by a read that proves it happened. A schema check confirms that the data has the right fields and types; it has no opinion on whether those values are true. Shape and truth are separate questions, and the test has to ask both.
Third, HTTP methods come with manners. A safe method such as GET should change nothing. An idempotent method such as PUT or DELETE lands in the same state however many times it is repeated. POST makes no such promise, so a nervous retry can order two of everything, which is why some APIs accept an idempotency key that lets the server recognize the repeat.
Now the surprise. A GraphQL API may reply with a cheerful 200 while the query partly failed, because HTTP has no status code for "sort of". The bad news sits in an errors field in the body, where a status-only test will never look. Likewise, the top risk on the OWASP API security list is not an exotic exploit. It is asking for someone else's record by changing an ID, and getting it. Catching that takes a test with two users, not one.
There is also a whole supporting cast: mock servers that stand in for other services, contract tests that keep two teams from surprising each other, and generators that invent thousands of inputs from an API description. They are useful, and each proves only what it actually exercises.
For the full picture, read the Intro for how a test is built and what each assertion can catch. The Cheatsheet is the lookup table for status codes, method rules, and tool syntax. The Practice tab shows the commands, and the Exercise has you run a small, working suite in containers on your own machine.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://www.rfc-editor.org/rfc/rfc9110.html
Supports
- Status code classes and the meanings of 400, 401, 403, 404, 409, 412, 415, and 422
- Safe methods GET, HEAD, OPTIONS, TRACE and idempotent methods including PUT and DELETE
- ETag as a validator and If-Match conditional requests
- Publication in June 2022 as the consolidated HTTP semantics specification
- https://www.rfc-editor.org/rfc/rfc6585.html
Supports
- 429 Too Many Requests for rate limiting, with an optional Retry-After header
- https://www.rfc-editor.org/rfc/rfc5789.html
Supports
- PATCH is neither safe nor idempotent, but can be issued idempotently
- https://www.rfc-editor.org/rfc/rfc9457.html
Supports
- Problem details media type application/problem+json and the type, title, status, detail, instance members
- RFC 9457 obsoletes RFC 7807, published July 2023
- https://json-schema.org/understanding-json-schema/reference/object
Supports
- properties, required, and additionalProperties allowing extra properties by default
- https://spec.openapis.org/oas/latest.html
Supports
- OpenAPI as a machine-readable API description with schemas and links
- Swagger 1.0 on 2011-08-10, OpenAPI 3.0.0 on 2017-07-26, OpenAPI 3.1.0 on 2021-02-15
- https://learn.openapis.org/specification/links.html
Supports
- Link objects describe relationships between operations with operationId and runtime expressions
- https://learn.openapis.org/upgrading/v3.0-to-v3.1.html
Supports
- OpenAPI 3.1 full compatibility with JSON Schema Draft 2020-12, nullable made redundant by type arrays, examples replacing example
- https://api-security.owasp.org/editions/2023/en/0x11-t10
Supports
- The ten API1 to API10 2023 risk names and descriptions
- https://api-security.owasp.org/editions/2023/en/0x04-release-notes
Supports
- 2023 edition merged excessive data exposure and mass assignment and added new categories
- https://api-security.owasp.org/editions/2019/en/0x00-header
Supports
- First OWASP API Security Top 10 released on 2019-05-29
- https://api-security.owasp.org/
Supports
- OWASP API Security Project home and Creative Commons Attribution-ShareAlike 4.0 licensing
- https://graphql.org/learn/serving-over-http/
Supports
- Single GraphQL endpoint, POST body with query, variables, operationName
- Response data and errors members, 2xx status when data is non-null even with errors
- https://grpc.io/docs/guides/status-codes/
Supports
- gRPC status codes OK 0 through UNAUTHENTICATED 16 and their meanings
- https://docs.stripe.com/api/idempotent_requests
Supports
- Idempotency-Key header on POST, saved first result including 500 errors, parameter mismatch rejection, keys pruned after at least 24 hours
- https://martinfowler.com/articles/practical-test-pyramid.html
Supports
- Test pyramid layers and very few end-to-end tests
- Consumer-driven contracts and stubbing external services with WireMock
- https://learning.postman.com/docs/tests-and-scripts/write-scripts/test-scripts/
Supports
- pm.test, pm.expect with Chai syntax, pm.response.to.have.status, post-response scripts
- https://learning.postman.com/docs/postman-cli/postman-cli-collections/
Supports
- postman collection run syntax, -e environment, -r reporters cli json junit html, --suppress-exit-code
- https://github.com/postmanlabs/newman
Supports
- Newman as the command-line collection runner, newman run with -e, reporters, non-zero exit on failure, --suppress-exit-code
- https://www.postman.com/company/about-postman/
Supports
- Postman started as a side project to simplify API testing
- https://blog.postman.com/how-we-built-postman-product-and-company/
Supports
- Postman published on the Chrome Web Store and first announced in a 2012 Stack Overflow answer
- https://blog.postman.com/timeline-postman-journey-to-api-platform/
Supports
- 2014 collections become a standardized way to define, save, and run API requests
- https://hurl.dev/
Supports
- Hurl plain-text format, captures, asserts, request chaining, built on libcurl, JUnit TAP HTML reports
- https://hurl.dev/docs/running-tests.html
Supports
- --test mode, --report-junit, --report-html, --report-json, --report-tap, --variable
- https://hurl.dev/docs/asserting-response.html
Supports
- Implicit status and header asserts, Asserts and Captures sections, jsonpath predicates
- https://hurl.dev/docs/request.html
Supports
- BasicAuth section in Hurl requests
- https://hurl.dev/docs/installation.html
Supports
- Hurl Docker image ghcr.io/orange-opensource/hurl
- https://github.com/mccutchen/go-httpbin
Supports
- go-httpbin container image ghcr.io/mccutchen/go-httpbin, default port 8080, echo, uuid, and basic-auth endpoints
- https://github.com/rest-assured/rest-assured/wiki/Usage
Supports
- given when then syntax, statusCode and body assertions, matchesJsonSchemaInClasspath with json-schema-validator
- https://docs.karatelabs.io/
Supports
- Karate API, UI, performance, and mocks in one syntax, Gherkin-style feature example with match
- https://karatelabs.io/company
Supports
- Karate released as open source on 2017-02-09 by Peter Thomas at Intuit
- https://schemathesis.readthedocs.io/en/stable/
Supports
- Tests generated from OpenAPI and GraphQL schemas, stateful workflows via links, curl reproducers
- https://schemathesis.readthedocs.io/en/stable/reference/cli/
Supports
- schemathesis run with --url, -H, --checks, --phases, --report junit, exit codes 0 1 2
- https://schemathesis.readthedocs.io/en/stable/reference/checks/
Supports
- Descriptions of not_a_server_error, status_code_conformance, content_type_conformance, response_schema_conformance, ignored_auth
- https://docs.pact.io/
Supports
- Contract testing definition, consumer-driven contracts, pact files, provider verification, Pact Broker
- https://docs.pact.io/history
Supports
- Pact written at realestate.com.au in 2013
- https://docs.pact.io/consumer/contract_tests_not_functional_tests
Supports
- Contract tests should not test provider business rules; over-specified validation breaks on harmless changes
- https://wiremock.org/docs/stubbing/
Supports
- Canned responses for matching requests, JSON mappings directory, stub priority, request verification
- https://docs.stoplight.io/docs/prism/674b27b261c3c-prism-overview
Supports
- prism mock and prism proxy commands
- https://docs.stoplight.io/docs/prism/f51bcc80a02db-installation
Supports
- Prism CLI install and default port 4010
- https://github.com/stoplightio/prism
Supports
- prism mock from an OpenAPI document, prism proxy to an upstream URL, --errors flag in validation proxy guide
- https://ics.uci.edu/~fielding/pubs/dissertation/top.htm
Supports
- REST defined in Roy Fielding's 2000 dissertation at UC Irvine
- https://arxiv.org/abs/2204.08348
Supports
- Ten REST API test generators on twenty services reached relatively low coverage; constrained inputs and inter-request dependencies were common limitations
- https://curl.se/docs/manpage.html
Supports
- curl -i, -s, -o, -w with %{http_code}, -H options
- https://jqlang.org/manual/
Supports
- jq -e exit status based on the last output
- https://github.com/sindresorhus/awesome
Supports
- Discovery index pointing to awesome-rest and awesome-testing
- https://github.com/marmelab/awesome-rest
Supports
- Testing and mocking listings for HTTPie, jq, REST Assured, Schemathesis, Step CI, Postman, SoapUI, json-server, httpbin, MockServer, Mockoon
- https://github.com/TheJambo/awesome-testing
Supports
- API Testing and Service Virtualization listings for Bruno, Keploy, Swagger Coverage Tool, MockServer, WireMock
- https://stepci.com/
Supports
- Step CI API testing framework for REST, GraphQL, and gRPC
- https://keploy.io/
Supports
- Keploy generates API test cases with dependency mocks
- https://www.usebruno.com/
Supports
- Bruno open-source, Git-friendly API client for REST, GraphQL, and gRPC
- https://github.com/usebruno/bruno
Supports
- Bruno stores collections as plain-text files in a filesystem folder
- https://insomnia.rest/
Supports
- Insomnia API client from Kong with open-source and free tiers
- https://github.com/Kong/insomnia
Supports
- Insomnia storage in a local vault, cloud sync, or Git repository
- https://github.com/hoppscotch/hoppscotch
Supports
- Hoppscotch open-source API client with self-host documentation
- https://www.soapui.org/
Supports
- SoapUI Open Source functional API testing for REST and SOAP
- https://smartbear.com/product/ready-api/
Supports
- ReadyAPI functional testing, performance, and API virtualization
- https://katalon.com/
Supports
- Katalon test automation for web, mobile, API, and desktop with a free start
- https://pactflow.io/
Supports
- PactFlow hosted contract testing built around Pact, with a free Starter plan
- https://wiremock.org/
Supports
- WireMock open-source engine and WireMock Cloud free tier
- https://stoplight.io/open-source/prism
Supports
- Prism open-source HTTP mock server
- https://grafana.com/docs/k6/latest/using-k6/thresholds/
Supports
- k6 thresholds as pass or fail criteria for load tests
- https://www.zaproxy.org/
Supports
- ZAP open-source web application and API security scanner
- https://httpie.io/
Supports
- HTTPie command-line HTTP client
- https://www.mock-server.com/
Supports
- MockServer mocking, record and replay, request verification, fault injection
- https://mockoon.com/
Supports
- Mockoon local mock APIs without remote deployment or account
- https://github.com/typicode/json-server
Supports
- json-server REST API from fixture files
- https://httpbin.org/
Supports
- httpbin HTTP request and response service
- https://github.com/Nikita-Filonov/swagger-coverage-tool
Supports
- API test coverage measured against Swagger or OpenAPI documentation
