Log in
← Blog

Pairwise Testing: Cover Hundreds of Combinations With a Dozen Tests

When options multiply, testing every combination is impossible. Pairwise testing covers every pair of values in a small set of tests, and catches most interaction bugs.

Vihari Panyam

August 18, 2026 · 4 min read

A desk with a monitor, a laptop and a phone side by side

In short

  • Pairwise (all-pairs) testing picks a small set of tests in which every pair of values from any two parameters appears at least once.
  • NIST research found most software failures are triggered by one or two interacting parameters, so covering every pair catches most interaction bugs.
  • Free tools like Microsoft’s PICT generate the test set for you; you add the constraints and the boundary values.
On this page

Your web app supports 4 browsers, 3 operating systems, 5 languages, 2 account types and 3 payment methods. Testing every combination means 4 × 3 × 5 × 2 × 3 = 360 test runs for a single checkout scenario. Nobody does that. So teams usually test a few combinations that feel important and hope the rest behave.

Pairwise testing replaces hope with a method. In this example, about 15 to 20 carefully chosen tests cover every pair of values, and that turns out to be where most interaction bugs live.

01Why pairs?

Most bugs are triggered by a single value (the German language breaks a layout) or by two values together (Safari plus PayPal fails). Far fewer need three or more specific values at once. NIST studied this from 1999 to 2004 and found that most failures were caused by one or two parameters, with fewer and fewer caused by three or more, and in the systems it studied, failures were triggered by interactions of between 1 and 6 variables.

So if every pair of values appears together in at least one test, you catch every single-value bug and every two-value bug, with a fraction of the tests.

02A small worked example

Three parameters with two values each: browser (Chrome, Safari), account (Guest, Member), payment (Card, PayPal). All combinations: 2 × 2 × 2 = 8. A pairwise set needs only 4:

TestBrowserAccountPayment
1ChromeGuestCard
2ChromeMemberPayPal
3SafariGuestPayPal
4SafariMemberCard

Check any two columns: every pair of values appears. Chrome + Guest, Chrome + Member, Safari + Guest, Safari + Member; Guest + Card, Guest + PayPal, and so on. Half the tests, every pair covered. The saving grows fast as parameters are added: the 360-combination example above drops to roughly a twentieth.

03Generating the set with a tool

You don’t build these by hand beyond toy examples. Microsoft’s free, open-source PICT tool takes a plain text model and prints the tests.

# checkout.txt  (run: pict checkout.txt)
Browser:  Chrome, Firefox, Safari, Edge
OS:       Windows, macOS, Android
Language: English, German, Japanese, Arabic, Spanish
Account:  Guest, Member
Payment:  Card, PayPal, ApplePay

# Constraints: combinations that can't exist
IF [Browser] = "Safari" THEN [OS] = "macOS";
IF [Payment] = "ApplePay" THEN [Browser] = "Safari";
text

04Reading and using the generated set

The tool prints one row per test. Each row is a complete configuration to run the checkout scenario on: a browser, an operating system, a language, an account type and a payment method. A few rows from a set like this:

TestBrowserOSLanguageAccountPayment
1ChromeWindowsGermanMemberPayPal
2SafarimacOSArabicGuestApplePay
3FirefoxAndroidJapaneseMemberCard
4EdgeWindowsSpanishGuestCard

Run the same scenario once per row. When a row fails, you have a short list of suspects: the values in that row, and especially the pairs in it that no other passing row shares. Re-run with one value changed at a time and the culprit usually shows up within two or three runs.

Keep the model file next to the tests. When a new browser or payment method is added, change the model and regenerate, instead of trying to edit the table by hand. PICT can also mark values as invalid (it uses a ~ prefix), so each test includes at most one bad value and a failure points at it clearly.

05Where pairwise works well

  • Configuration testing: browsers, devices, operating systems, screen sizes.
  • Settings and options: a form with many independent dropdowns, a report with many filters.
  • Feature flags: which flags on together still work.
  • API parameters: optional query parameters that can be combined.

06Where it doesn’t

  • Known high-risk combinations. If you know payment + currency + country together decide the tax, test those combinations fully with a decision table.
  • Values inside each parameter. Pairwise picks combinations, not values. You still need boundary values for amounts and lengths.
  • Sequences. The order of actions matters for workflows; that’s state transition testing.
  • Safety-critical systems, where 3-way or higher coverage (also supported by PICT and NIST’s ACTS tool) is worth the extra tests.

Pairwise testing checklist

  • Parameters and their values are listed, including invalid or unusual values where relevant
  • Impossible combinations are written as constraints
  • The set is generated by a tool, not by hand
  • Known high-risk combinations are added explicitly
  • Numeric values inside each test use boundary values
  • Higher-strength coverage is used where failure is very costly

Frequently asked questions

What is pairwise testing?

A combinatorial test design technique that selects a small set of test cases in which every possible pair of values, across any two parameters, appears together at least once.

How many test cases does pairwise testing need?

Far fewer than all combinations. It depends on the number of values per parameter; roughly, it grows with the product of the two largest parameter sizes rather than with every parameter multiplied together.

Is pairwise testing enough?

It catches most interaction bugs involving one or two parameters. For combinations you already know are risky, or for safety-critical software, add those combinations explicitly or use 3-way coverage.

What tools generate pairwise tests?

Microsoft PICT (free, open source) and NIST’s ACTS are widely used. Several test management tools also include pairwise generators.

What is the difference between pairwise and orthogonal array testing?

Both aim to cover pairs efficiently. Orthogonal arrays come from statistics and cover each pair the same number of times; pairwise tools simply guarantee each pair appears at least once, usually with fewer tests.

Written by

Vihari Panyam

QualityAIQ

Sources and further reading

  1. NIST: Automated Combinatorial Testing for Software
  2. Microsoft PICT on GitHub
  3. ISTQB Glossary: pairwise testing

More in Test Design