MyVar.dev
Technologies and Software Engineering
contract-testing

Contract Testing Ensuring Service Compatibility and Faster Feedback

Contract testing validates service interfaces and message structure for compatibility. It provides early detection and faster feedback in CI CD.

Overview

Contract testing verifies that interacting services adhere to a shared interface specification, ensuring compatibility without extensive end-to-end integration tests. It focuses on message exchange and structure, rather than deep functional behavior or side effects.

Key Insights

  • Interface Focus: Contract tests validate the interface between services, not internal functionality.
  • Consumer-Driven: Consumers define their expectations, driving the contract (e.g., in Pact).
  • Producer Verification: Producers verify their service against these consumer-defined contracts to ensure compatibility.
  • CI/CD Integration: Tools like Pact Broker integrate contract testing into CI/CD pipelines, enabling faster, independent deployments.
  • Faster Feedback: Contract tests execute quicker than traditional integration tests, providing earlier feedback on breaking changes.
  • Bi-Directional Contract Testing (BDCT): A schema-based approach that statically compares consumer expectations against provider capabilities, decoupling verification from execution.

Technical Details

Understanding Contract Testing

Contract testing ensures service call inputs and outputs contain required attributes, and that response latency and throughput meet acceptable limits. Unlike traditional integration tests focusing on end-to-end functionality, contract tests isolate applications to check message conformity to a shared understanding, or “contract.”

Contract Tests vs. Integration Tests

  • Contract Tests: Focus on the interface (messages, schemas) between a consumer and a provider. They validate attribute presence and basic performance, not deep internal behavior.
  • Integration Tests: Focus on functionality across multiple services, ensuring the entire system works as expected, including side effects like database entries.

Benefits of Contract Testing

  • Early Detection: Identify interface incompatibilities early in the development cycle.
  • Independent Deployment: Enables independent service releases with confidence.
  • Consumer Impact Awareness: Producers understand their changes’ impact on consumers.
  • Performance: Quicker execution times compared to full integration test suites.

Pact: Consumer-Driven Contracts

Pact is a contract testing tool that facilitates consumer-driven contracts. Consumers define their expectations of a provider, which the provider then verifies.

How Pact Works

Pact utilizes a record-and-replay mechanism:

  1. Consumer Side: During consumer unit tests, requests made to a Pact mock provider are recorded into a Pact file (the consumer contract), along with expected responses. Adapters like WireMock or MSW can substitute the Pact mock provider.
  2. Provider Side: A Pact verification task replays these recorded requests against the real provider service. Actual responses are compared against the expected responses in the Pact file.
  3. Verification: If responses match, the contract is verified, confirming the real provider fulfills consumer expectations.

Pact Workflow

Pact is consumer-driven, meaning the consumer defines the contract. Consumer tests establish a mock server based on the defined pact, allowing consumer code to be tested against it. The producer then takes the generated Pact file and verifies its actual service against the contract. Only the producer can truly verify service adherence to defined requests and responses.

Pact Broker: Orchestrating Contracts

The Pact Broker is an application for sharing consumer-driven contracts and their verification results. It centralizes the contract testing workflow, especially critical in microservice architectures.

Purpose and Features

The Pact Broker acts as a central repository for Pact files, facilitating:

  • Contract Sharing: Consumers publish pacts; providers retrieve them.
  • Version Management: Tracks pact versions, allowing multiple application versions to share the same pact if unchanged.
  • Backward Compatibility: Ensures new provider versions remain compatible with existing consumers.
  • CI/CD Integration: Automates contract testing in CI/CD pipelines, enabling independent service deployment.
  • Visibility: Visualizes and documents service relationships through auto-generated diagrams.
  • Webhooks: Triggers actions (e.g., provider builds) upon pact changes or verification results.
  • Compatibility Matrix: Provides a matrix of compatible consumer and provider versions.
  • CLI: Offers a command-line interface for common Pact workflow tasks.

Pact Broker Workflow

  1. Consumer Project Execution:
    • The consumer project runs tests using a Pact library or adapter (e.g., WireMock, MSW) for a mock service.
    • The mock service generates a Pact file (the consumer contract) by recording requests and expected responses.
    • The generated Pact file publishes to the Pact Broker.
    • If pact content changes, a Pact Broker webhook can trigger a build of the relevant provider project for verification.
  2. Provider Project Verification:
    • The provider’s CI/CD pipeline includes a verification task configured to retrieve relevant pacts from the Pact Broker.
    • The provider build runs pact verification, replaying each request from the pacts against the real provider service and checking responses.
    • If verification fails, the build fails.
    • Verification results (pass/fail) publish back to the Pact Broker, informing the consumer team of compatibility.
    • If successful, provider deployment proceeds and records with Pact.
  3. Consumer Project Deployment:
    • The consumer’s CI/CD pipeline checks the Pact Broker for successful contract verification.
    • If the pact is successfully verified, consumer deployment proceeds.
    • The consumer records its deployment with Pact.

can_i_deploy Utility

The can_i_deploy command-line tool enhances deployment safety. It checks the Pact Broker for contract verification status. If a deployment would break an existing contract or if a contract remains unverified, the tool fails the pipeline, preventing broken releases. It typically integrates into CI/CD pipelines before deployment to each environment, with environment tags applied post-deployment.

Bi-Directional Contract Testing (BDCT): A Schema-Based Approach

Bi-Directional Contract Testing (BDCT) is a static contract testing method comparing consumer expectations and provider capabilities based on their schemas. This approach offers an alternative to Pact’s record-and-replay style, particularly useful in contract-first or provider-driven workflows. BDCT is a feature exclusive to PactFlow (a paid solution).

BDCT vs. Pact

  • Pact: Employs a record-and-replay mechanism using specification by example. Consumer tests interact with a mock, generating a pact file later replayed against the actual provider.
  • BDCT: Utilizes a static schema comparison. The consumer contract (expectations) compares against the provider contract (capabilities), typically an OpenAPI specification. The consumer contract is never replayed against the provider codebase.

BDCT Workflow

BDCT simplifies the workflow by decoupling verification from live interaction:

  1. Consumer Side:
    • Consumer tests its behavior against a mock (e.g., WireMock, MSW).
    • A consumer contract (often a Pact file or generated from the mock) produces, capturing interaction expectations.
    • This consumer contract uploads to PactFlow.
  2. Provider Side:
    • The provider defines its capabilities in a provider contract (e.g., an OpenAPI specification), hand-written or generated from source code.
    • Provider code tests for adherence to its own provider contract using API testing tools (e.g., RestAssured, Dredd, Postman).
    • This provider contract uploads to PactFlow.
  3. Static Comparison:
    • PactFlow statically compares the uploaded consumer contract with the provider contract.
    • It verifies that the consumer’s expectations are a valid subset of the provider’s advertised capabilities (schema comparison).
    • This comparison ensures continuous API compliance.

Advantages of BDCT

  • Low Barrier to Entry: Leverages existing mocks and tools.
  • Faster Scaling: Reuses current processes and tooling.
  • Decoupled Workflow: Eliminates replaying consumer contracts against the provider codebase, simplifying integration.
  • Wider Audience: Appeals to developers, QA, and SDETs, supporting both white-box and black-box testing.

Core Concepts in Contract Testing

Consumer Contract

A consumer contract is a collection of interactions describing how a consumer expects a provider to behave. Each consumer typically has a unique contract for each provider it interacts with, detailing required requests and expected responses.

Provider Contract

A provider contract specifies the full capability of the provider. This can take various forms, such as an OpenAPI document, GraphQL schema, SOAP XSD, or Protobuf definition. It details all services and data structures a provider offers.

Distinguishing Contract and Functional Tests

  • Contract Tests: Primarily focus on the schemas and structure of messages exchanged between services.
  • Functional Tests: Go beyond structure to ensure correct business logic execution and that expected side effects (e.g., a new database entry, status changes) have occurred.

Conclusion

Contract testing, especially with tools like Pact and Pact Broker, provides a robust strategy for managing compatibility in distributed systems. By focusing on interfaces and enabling consumer-driven approaches, it allows teams to achieve faster feedback, independent deployments, and greater confidence in their microservice architectures. Bi-Directional Contract Testing offers an advanced, schema-based alternative for scenarios requiring static verification, further enhancing API compliance.

Tags

Search

Find technical notes