Prioneer

Free Online Stack Ranking Tool

Kostenloses Online Stack Ranking Tool

Rank ideas, customer problems, or choice options collectivly or by yourself.

Stack Ranking Tool

Rank a list of alternatives using pairwise comparison.

Pairwise Ranking

Gather insights about your audiences preferences with a ranking survey.

Ranking Surveys

Modern, intutive surveys to research the best pictures, logos or product designs.

Image Ranking Surveys

Use group rankings to quickly rank items with your team, friends and family.

Group Decision Tool

Prioritize Jira issues using RICE, ICE, AARRR, or a custom prioritization framework.

Jira Backlog Prioritization

Products

Blog

md:![Preview of a pairwise ranking survey and settings.](https://prioneer.io/cloudassets/s1/prioneer/help-center/article-406887922440-pairwise-ranking-survey-preview.png)

## Create a survey: step-by-step

1. Go to your [Rankings view](https://prioneer.io/home) in Prioneer.
2. Click on "Start a new Ranking"
3. You can now pick from several Ranking templates. Select "Pairwise Ranking" as your template.
4. In the next step, you can preconfigure the most important settings. You can adjust the settings or leave them as they are. You can change all settings later.
5. Your new Ranking will be opened. Follow the instructions on your screen, such as setting the Ranking name.
6. You are now in your Ranking. Inside the "Create" tab, you can set the Ranking question and add your choice options.
7. Set your Ranking question.

    Set the Ranking question as expressive as possible. The golden rule is that the better your question is, the better your results will be because participants will consider the question as guiding for their decision.
A good Ranking question should be related to a specific goal. Good examples are: "Which one is better to raise our number of signups?"
"Which one is better to increase brand awareness?". Think of what you want to achieve.
8. Add your choice options.

    Each choice option can have a name, a description, and image attachments. The name is the primary information a participant will view when answering your ranking survey, 
so make sure to write the choice option name as expressive as possible. Images will be displayed as an attachment on a choice option card during the pairwise comparison of the choice options (unless you change your Ranking to be an image ranking survey. In that case, the image will be shown "fullcard"-size as primary information).
9. Your Ranking survey is now ready to be shared with participants through a link (or other sharing options). You can either stay with the default settings (viable to be used immediately) or adjust some settings. However, you do not have to adjust any settings. To share your survey with participants, click on the Share button on the top right corner of the screen and select your preferred sharing method.

## Editing advanced settings, permissions, and design

To adjust settings, go to the "Configure" section inside of your Ranking. You can see a preview of your pairwise ranking survey and adjust settings for your ranking.

## How many pairs must a participant answer in a pairwise comparison survey?

The number of pairs that will be in the head-to-head comparison will depend on the number of choice options you have.
In theory, you (or a participant) need to compare each choice option with each other to select a preference.
However, this can make the process of answering very long.
When you create a ranking survey in Prioneer, you don't have to worry about the number of choice options: 
Prioneer uses a smart algorithm that selects only relevant pairs when a participant starts answering.
With each pair that a participant answers, Prioneer will calculate on-the-fly the next most relevant pair to compare
so that Prioneer can determine the unique preferences of the participant.
Because of that, a particpant does not have to answer all pairs (the Ranking survey will show the participant that he is finished).
This algorithm, which we also call speed mode, is unique to Prioneer.

## Frequent use cases

### Add images to a pairwise ranking survey
To attach an image to a choice option, follow these steps: go to the "Create" section of your Ranking. Click on the choice option that you want to attach an image to.
Next to the list of choice options, a details view will be shown in which you can edit fields related to your choice option.
Click on the attachment icon. You can now select an image from your file system that you want to upload.


### Bulk create options

You can create options in bulk by clicking the "Add multiple options" button beneath the list of options.
A textarea will appear on the screen in which you can add multiple options at once by adding them into the textarea, each item on its own row.

### Create a pairwise ranking survey from a CSV file or excel spreadsheet
To create a survey based on options you have saved inside a spreadsheet, go to your spreadsheet. Select the column with the names, and copy the names into your OS clipboard.
In Prioneer, go to the "Create" section inside a Ranking (your Ranking must have been created before) and click the "Add multiple options" button beneath the list of options.
You can now paste all names that you copied before from your spreadsheet into the textarea that is shown on the screen.
Usually, each row from your sheet will have its own row inside the textarea, thus each row will be recognized as a separate choice option and generated accordingly as soon as you confirm the action.

### Bulk upload images to a Ranking survey

You can upload multiple images at once. When you use the image bulk upload, our system will generate one choice option per image
and attach the image to each choice option.
If you want to attach an image to an existing option instead, please click on the choice option and attach an image via the details dialog that is shown next to your list of choice options. All images that you attach through the details dialog will be added to the particular choice option.

To perform bulk uploading images, click the "Add multiple options" button beneath the list of options.
You will see multiple possible actions. Click on the button "Images". You will be able to select images for upload now.

Learn how to create a pairwise ranking survey, how to write a good question, and how many choice options you should have.

How to create a pairwise ranking survey

md:![A gauge with the sections low, medium, and high, representing a high answer confidence.](https://prioneer.io/cloudassets/s1/prioneer/help-center/article-406887922439-confidence.jpg)

## What is the answering confidence?

The confidence of a participant is a value between 0-100%. Our system calculates the confidence of a participant while he/she gives answers. The more answers a participant gives, the higher is his/her Rankings confidence. It indicates how certain our system is of the participant's individual preferences,
and how high the certainty is for the unique ranking (rank 1,2,3...) of all choice options is.

The confidence of a participant is calculated in real-time while the participant answers the choice options of a pairwise comparison.
As long as the confidence is low, our ranking system will not be able to create a ranking of all options with a unique rank per choice options.
Only when the participant moves on through the pairs and keeps answering, the confidence increases and a clear ranking (such as Project A = Rank 1, Project B = Rank 5, Project C = Rank 3) will be established by our tool. 

The confidence is displayed in the progress bar to the participant during the ranking survey (for example: 50% progress bar filled represents 50% confidence).

## With Prioneers speed mode, Ranking surveys are answered faster
Based on the answering confidence, a participant can finish a pairwise comparison faster because the participant does not need to compare all possible pairs.
Prioneer can calculate transitive preferences based on a participants answers (without the participant having to answer all pairs) and create a unique list of a participant's priorities with high confidence.
The answering process will stop for the participant as soon as the confidence is high enough to be certain of his preferences.

### Finish threshold
With the default ranking settings, the participant can continue to answer if he likes, which will make his ranking quality finer. In the settings, you can modify the conditions that control when the ranking answering process should stop for the user.

Learn what the answering confidence is and how it speeds up the answering for a participant.

What is the answering confidence in a pairwise ranking survey

md:![Participants of a pairwise ranking, and the agreement about the priority. The team is aligned with the priority.](https://prioneer.io/cloudassets/s1/prioneer/help-center/article-406887922438-priority-agreement.png)

## What's the priority agreement?

As you view the results of a Pairwise Ranking, you will come across the so-called **agreement**.

The agreement is a statistical number and tells you how much all people who responded to your Ranking agree on the priority of a choice option.
(You could also say: how much they agree about the *rank* or *importance* of a choice option.)
In ranking results, each choice option has an agreement.

The agreement is implicit because respondents to the ranking survey cannot submit their agreement about the priorities but it is a scientific number that is calculated automatically by our system based on their responses.

## What's the agreement good for?

The agreement helps you to: 
* make data-based decisions
* identify misunderstandings
* raise the right discussions
* align your team on the priorities
* understand confidence of priorities


## Understanding high and low agreements

If the agreement percentage of a choice option is high, e.g. > 90%, it means that many people ranked the choice option similarly important within their individual Ranking.
For instance, if in person Bob's responses the option A is rank #1, in Anne's responses the option A is rank #1, and in Peter's responses the option A is rank #1,
then the agreement is 100%.

<div style="display: flex; justify-content: center;">
| Choice Option     | Bob's Rank    | Anne's Rank   | Peter's Rank    |   Agreement
| -----------       | -----------   | -----------   | -----------     | -----------   
| A                 | #1             | #1             | #1               |  100%
</div>
<div style="display: flex; justify-content: center;margin-bottom: 40px;">100% agreement because it is rank #1 for everyone.</div>

If however in person Bob's responses the option A is *only rank #2*, in Anne's responses the option A is rank #1, and in Peter's responses the option A is rank #1,
then the agreement will be around between 90%-99% (the exact agreement depends on the total number of choice options, actual replies and the settings of the Ranking).

<div className="alert-notice" style="margin-bottom: 50px;">
Please note: the agreement numbers in all following examples will differ for your own Rankings! The exact agreement always depends
on multiple factors such as the number of choice options, the exact replies your Ranking received, and the settings of the particular Ranking.
</div>


<div style="display: flex; justify-content: center;">
| Choice Option     | Bob's Rank    | Anne's Rank   | Peter's Rank    |   Agreement
| -----------       | -----------   | -----------   | -----------     | -----------   
| A                 | #2             | #1             | #1               |  95%
</div>
<div style="display: flex; justify-content: center;margin-bottom: 40px;">High agreement among respondents, but not 100%.</div>


If the agreement percentage is low, it means that many people ranked the choice option very differently in their individual rankings. For instance, let's assume in a Ranking survey with 5 choice options, for Bob the option A is rank #1,
for Anne the option A is rank #3, and for Peter the option A is rank #5.
In this situation, the agreement would be a low percentage number, e.g. 2%.  

<div style="display: flex; justify-content: center;">
| Choice Option     | Bob's Rank    | Anne's Rank   | Peter's Rank    |   Agreement
| -----------       | -----------   | -----------   | -----------     | -----------   
| A                 | #1             | #3             | #5               |  2%
</div>
<div style="display: flex; justify-content: center;margin-bottom: 40px;">Low/no agreement</div>

### Why is a choice option on rank #1 but has a low agreement?

Sometimes you might face the situation that the most important choice options or one of them has a low agreement.
E.g. something is on rank #1 in the results of your ranking survey but the agreement is just 50%.
What does that mean?

It means that no other choice option was as important as that choice option among all people who replied (leads to rank #1 in total)
but there were still people who did not rank that particular choice option as the most important (leads to low agreement despite rank #1).

For better understanding, let's assume these two situations:

If a choice option is on rank #1 and the agreement is 100%, the choice option got the highest priority in everyone's individual ranking (or in other words: it is rank #1 in everyones individual ranking).

If a choice option is on rank #1 and the agreement is 50%, the choice option won the highest priority in many individual rankings,
but not in all individual rankings.
Nevertheless, the choice option ranked position #1 because no other choice option is more important after calculating all individual rankings into one combined ranking.

<div style="display: flex; justify-content: center;">
| Choice Option     | Bob's Rank    | Anne's Rank   | Peter's Rank  | Total Rank  |   Agreement
| -----------       | -----------   | -----------   | -----------     | -----------     | -----------   
| A                 | #1             | #1            | #4           | #1   |  70%
</div>
<div style="display: flex; justify-content: center;margin-bottom: 40px;">A high position can have a low agreement</div>

### Why has a choice option 100% agreement but is not on rank #1?

A choice option can have 100% agreement on any rank.
For example, if everyone ranks choice option A as the worst choice option,
the choice option will land on the last position of the ranking with an agreement of 100%.

Similarly, if a choice option is ranked by everyone so that its position ends up being between #4 - #6 across all individual rankings,
then the agreement will also be close to 100% (the exact agreement depends not only on the position but also on other factors as the number of ranking options.)  

<div style="display: flex; justify-content: center;">
| Choice Option     | Bob's Rank    | Anne's Rank   | Peter's Rank  | Total Rank   |   Agreement
| -----------       | -----------   | -----------   | -----------   | -----------    | -----------   
| A                 | #10             | #10            | #10           | #10   |  100%
</div>
<div style="display: flex; justify-content: center;margin-bottom: 40px;">The last position can have a high agreement (assuming 10 choice options).</div>

<div style="display: flex; justify-content: center;">
| Choice Option     | Bob's Rank    | Anne's Rank   | Peter's Rank  | Total Rank   |   Agreement
| -----------       | -----------   | -----------   | -----------   | -----------    | -----------   
| A                 | #4             | #5            | #6           | #5   |  99%
</div>
<div style="display: flex; justify-content: center;margin-bottom: 40px;">A medium position can have a high agreement (assuming 10 choice options).</div>

Learn what the agreement is and how it helps you to make better data-driven decisions.

How to use the priority agreement

Get started - it's free

Decide the right things, with the right people.
Rank choice options from most important to least important
and <markerHighlight>achieve your goals 10x faster</markerHighlight>.

A simpler way for everyone to
make great decisions

Cesar Daniel Valdivia Vergara

Your tool has been game changing for our work.

Use the prioritization power of proven management & product decision-making techniques without fiddling in spreadsheets.

Want to decide together with others? No problem.
Add people to your Prioneer workspace or share a guest link. Everyone can submit their own opinion and Prioneer shows you the overall opinion of the group.

Invite anonymously
or with login

You control who can
see results

Asynchronous voting
without meeting

Project Management Consultant

I was looking for a way to make people do individual scorings […] to <b600>avoid group thinking</b600> and make their own assessments first, and then we'd make changes after that in discussion. Thank you!

Stephan Eyl

The team rating allows us to <b600>quickly prioritize as a team</b600>, without having to discuss each project one by one. We <b600>save a lot of time</b600>, and <b600>the entire team has clarity</b600> regarding which of the new projects we expect to have the greatest impact.

Group Product Manager

We are using Prioneer for customer needs discovery […]. I can’t thank you enough for your help with this! This is <b600>an incredible enhancement</b600> that will save us a lot of time.

Sign up now

Use Prioneer for:

Prioneer helps teams to make the right tradeoffs and to align on common priorities. Spend time and money on the right projects so that your business profits as a whole.

Get started now

Stephan Eyl
CEO Fanpage Karma

Since we started using Prioneer, we deliver much more often what our customers really have been waiting for.

Joshua Land
Founder Spreaducation

For a startup team, it's important not to waste time on things that have no impact. With Prioneer everyone is on the same page about the most important things.

Anything you want. Use Prioneer to get rankings on ideas,
customer pain points, projects or anything else.

What can I stack rank with Prioneer?

Don't let a winning team's time be used up by meaningless projects.

Earn applause for delivering
things that matter

Get started

Get started completely free.

Ready to prioritize the right things?

Start prioritizing with a free template.

Article: Pairwise comparison method explained

Prioneer is a stack ranking tool. Get people's priorities ·  Research customer pain points · Make team decisions · Create beautiful ranking surveys in 1 minute!

2023-03-24

Start within minutes. Upgrade when you wish. Don't pay extra per user.

Take your decision-making to the next level.

Pricing - Prioneer

There is content that you provide to us through our services when using our services. This can be content such as workspace content or ranking content (e.g. ranking names, questions, item name, user name, guest name, your submitted rating value for an item/idee). Also, when you submit a rating value for an item (by rating an item/project/software feature through the rating feature), your rating value is stored together with your user ID or guest user ID or name. We collect and store this in order to provide this service. Please note that the ranking value that you submit can be visible to the creators/members of a ranking or the members of a workspace.

The data (including ideaspace names, items/ideas, guest user names, guest ranking and so on) is deleted when the ranking creators/members delete the particular workspace or ideaspace.

Effective Decision Making

How to create a pairwise ranking survey

What is the answering confidence in a pairwise ranking survey

What is management decision making

Pairwise comparison method explained

How to use the priority agreement

Prioritization: Why Teams Should Try Forced Ranking

Stack Ranking Template for Excel

5 Methods to Prioritize Tasks

RICE Prioritization for Jira