Prioritization:Pairwise comparison method explained

Making a decision can be challenging when having multiple options to select from. The pairwise comparison method (or also called pairwise ranking) is a prioritization method often used by leaders for effective decision making. In higher management this approach is often used to compare and define the best course of action.

This article gives you a brief introduction to the method, shows you how to calculate the number of pairs, and explains to you how you can make the process faster by reducing the number of pairs that you need to rate. Also you can calculate the number of possible combinations using the calculator included in this page.

What is the pairwise comparison method?

The pairwise comparison method lets you compare pairs of choice options in a “left-or-right” manner to determine your preferences. It is a simple method that can be applied for any kinds of choice options (potential projects, feature ideas, job applications, images) to generate a ranking of those options from most preferred option to least preferred option.

To perform a pairwise comparison, you compare two choice options at once and select the better choice option. After selecting the favorite option, you pick the next two choice options and again select your favorite one. Then you continue, until you have selected each preferred choice option from all possible combinations. For example, assuming you have the choice options A, B, C and D, you start by comparing A and B, then you do A and C, then A and D, then B and C, B and D, and so on.

It is well suited to understanding people's preferences and is therefore often used in surveys because survey participants are forced to select unique preferences. The results you get out of such a survey are therefore much more actionable: instead of just knowing what choice options a person likes, they reveal in which order of preference the person likes those choice options.

How many pairs: calculating the number of combinations

Combining the individual elements in a pairwise comparison will yield all possible pairs (or groups).

You can calculate the number of pairs you need to assess using the formula: (n*(n-1))/2

For example:

10 choice options will result in 45 pairs.

20 choice options will result in 190 pairs.

With more choice options, the number of pairs grows significantly. Applying the method with many choice options can be slow and cumbersome. However, you can modify the process to make the method easier, even with many choice options.

Doing a quicker pairwise comparison by reducing the number of pairs

If you add many choice options, the number of pairs can become quite large. However, there is a way to reduce the number of pairs you need to assess.

You can reduce the number of pairs through detecting transitive preferences between your assessments. For instance, if you rate A better than B, and you rate B better than C, then you do not need to rate the pair A/C because you can derive transitively that A is better C.

Calculating and using transitive preferences will reduce the number of pairs significantly and will shorten the time you need to perform the whole assessment. To fully evaluate choices that would normally take you 30 minutes, you will only need 5-10 minutes. The exact time it takes to rate all pairs depends on the answers that you give, because the ratings that you give will not always be equally consistent (A > B > C > D). E.g., if you rank (A > B > C > D) but decide at a later point in the direct comparison of D/A that D is better than A (simplified), then you will need to make additional ratings of pairs to create a clean ranking because your preferences are not unmistakably clear.

Calculating transitive preferences is powerful, but not trivial to do. Transitive winners of pairs can change each time you rate one more pair in the process. You need to recalculate all dependencies after each new pair that you have rated. Since this is impossible to do effectively on paper or in an excel sheet, pairwise comparison tools such as Prioneer exist. Prioneer does all calculations automatically for you. When you create a Ranking in Prioneer, you only have to answers rate the pairs that Prioneer asks you, and Prioneer does all the background work (like asking you for additional pairs if your preferences are unclear) and gives you the final ranking. Create a ranking here to see how easy it is to evaluate pairs.

Evaluating choice options using multiple criteria - multidimensional comparison

When evaluating your choice options to find your preferences, you will always have a overarching goal in mind. For example: “Which is the better course of action to take now?” “Which is the better marketing strategy right now?” “What product feature should we develop next?”.

This is one dimension (or criterion). But what if you want to consider multiple criteria? For example, when selecting the best course of action you prefer item A over item B because it has the higher impact, but at the same time you prefer B over A because it has a lower effort to be implemented.

In that situation you can do a multi-dimensional pairwise comparison and compare choice options according to multiple criteria.

See also other ways to prioritize.