What is the answering confidence in a pairwise ranking survey

What is the answering confidence?

The confidence of a participant is a value between 0-100%. Our system calculates the confidence of a participant while he/she gives answers. The more answers a participant gives, the higher is his/her Rankings confidence. It indicates how certain our system is of the participant's individual preferences, and how high the certainty is for the unique ranking (rank 1,2,3...) of all choice options is.

The confidence of a participant is calculated in real-time while the participant answers the choice options of a pairwise comparison. As long as the confidence is low, our ranking system will not be able to create a ranking of all options with a unique rank per choice options. Only when the participant moves on through the pairs and keeps answering, the confidence increases and a clear ranking (such as Project A = Rank 1, Project B = Rank 5, Project C = Rank 3) will be established by our tool.

The confidence is displayed in the progress bar to the participant during the ranking survey (for example: 50% progress bar filled represents 50% confidence).

With Prioneers speed mode, Ranking surveys are answered faster

Based on the answering confidence, a participant can finish a pairwise comparison faster because the participant does not need to compare all possible pairs. Prioneer can calculate transitive preferences based on a participants answers (without the participant having to answer all pairs) and create a unique list of a participant's priorities with high confidence. The answering process will stop for the participant as soon as the confidence is high enough to be certain of his preferences.

Finish threshold

With the default ranking settings, the participant can continue to answer if he likes, which will make his ranking quality finer. In the settings, you can modify the conditions that control when the ranking answering process should stop for the user.