Test Automation




Define

The most important first step of any API lifecycle is to make sure the operations around an API are properly defined, laying the foundation for being able to effectively design and bring an API to life, while also establishing a known place, or places to go to get all the information you need regarding each individual API, or groups of APIs. A little planning and organization at this early step of the API journey can go a long way towards ensuring the overall health and velocity of an API, and the applications and integrations that will depend on each internal, partner, or public API being delivered.

Team Workspace

Establishing and properly setting up a dedicated workspace for each API helps ensure there is always a single place to go to find everything that is happening with an API across it's entire lifecycle.

Team Members

Formerly defining who the team will be moving an API forward through all stages of it's lifecycle, providing a clear definition of who is responsible for each part of producing an API.

Github Repository

Having a dedicated Github repository for an API provides a single place where code and other artifacts can be managed, with a pipeline, issues, and other supporting elements an API will need to operate.

OpenAPI

The OpenAPI specification provides a common vocabulary for describing the surface area of request and response APIs, as well as webhooks, ensuring that API producers and consumers are on the same page when it comes to integration, but also helps stabilize multiple areas of the API lifecycle providing a contract that can be used for delivering documentation, mocks, testing, and more.

Test

A test-driven API lifecycle ensures that each API accomplishes the intended purpose it was developed for, providing manual and automated ways to ensure an API hasn't changed unexpectedly, is as performant as required, and meets the security expectations of everyone involved, helping establish a high quality of service consistently across all APIs.

Contract Testing

Contract tests can be derived from API artifacts like OpenAPI, JSON Schema, and GraphQL, and used to ensure there are no changes to what has been promised when an API was designed or developed, providing a repeatable test that can be run manually, on a schedule from multiple regions, or as part of a CI/CD pipeline to ensure contracts with consumers are upheld.

Monitor

All tests applied to an API should be monitored on a logical schedule and from relevant geographic regions, monitoring that APIs aren't breaking their contract, falling below their agreed upon service level agreement (SLA), or becoming a security risk, helping automate the quality of service across APIs in a way that allows teams to be as productive as possible.

Monitor

Monitoring any process across API operations defined as a collection, then bundled with any environment, setting a schedule for the execution of the collection, and viewing or publishing of the results to any other location, providing a very wide definition of what monitoring can mean across API operations.

Contract Testing Monitor

Monitors can be setup for all API contract testing, scheduling the run of contract testing when needed and configuring to run from the regions that matter most to business operations, making sure that each individual API contract is never breaking with consumers, making sure that each API is delivering as expected on a 24/7 schedule.

Contract Testing Results

The results of any contract testing run can be rendered as test results, presented using visualizer, and posted to existing APM or other destination that can be used as part of wider observability systems, allowing contract tests across all APIs to be centralized for understanding API performance at scale.

Build

Tests can be automated using a CI/CD pipeline, allowing for test to be executed every time a commit or pull request is made against a repo. Allowing tests to be run against any instance of an API, ensuring that any API being deployed has not broken it's contract, and the API does what is expected.

CI/CD Pipeline

Like other aspects of the software development lifecycle, APIs benefit from well defined, repeatable CI/CD pipelines to deploy, test, automate, and orchestrate throughout the API lifecycle, publishing APIs to development, staging, production, and other stages, but then also ensuring other elements like publishing of documentation, and tests are baked into the process.

Newman

Newman is an open source collection runner, allowing Postman collections to be executed as part of CI/CD pipeline runs, executing all of the API requests, pre-request, and post-request scripts, and providing results reporting that can be used as part of testing and automation.

Development Environments

Machine readable environments for APIs allow for abstracting away common elements of a API development environment from the definition of each API, allowing different environments to be paired with collections for each API at design, develop, and build time.

Production Environments

Machine readable environments for APIs allow for abstracting away common elements of an API production environments from the definition of each API, allowing different environments to be paired with collections for each API at design, develop, and build time.

Contract Testing

Contract tests can be derived from API artifacts like OpenAPI, JSON Schema, and GraphQL, and used to ensure there are no changes to what has been promised when an API was designed or developed, providing a repeatable test that can be run manually, on a schedule from multiple regions, or as part of a CI/CD pipeline to ensure contracts with consumers are upheld.

Contract Testing Results

The results of any contract testing run can be rendered as test results, presented using visualizer, and posted to existing APM or other destination that can be used as part of wider observability systems, allowing contract tests across all APIs to be centralized for understanding API performance at scale.

Observability

Tapping into the outputs available across API operations to understand what is happening with individual APIs throughout their lifecycle, but also in aggregate for domains, teams, and other logical groups, helping make API operations more visible in real time.

Activity

The changes made to any aspect of operations by team members, providing observability into when APIs, mock servers, documentation, testing, monitors, and other critical elements of API operations are changed, configured-- helping give a log of everything that happens at the operational level.

Reports

Visual reports that aggregate data from across operations, making APIs and the operations around them something that team members can see activity, hisotry, and other dimension of what is happening across API operations, allowing different views to be organized and presented via dashboards.

Contract Testing Monitor Report

A report that allows the one time or historical results of contract testing monitors to br visualized, providing visibility into whether or not an API contract has been broken at any point in time as defined the schedule of the contract testing monitor runs.

Application Performance Management (APM)

The monitoring and management of performance and availability of APIs, actively detecting and diagnosing performance problems to maintain an expected level of service, translating API operational metrics into business meaning value at scale across hundreds or thousands of internal or external APIs.

The End