Chapter 3: Voting
Chapter 3 is about voting. This is the stage at which a group comes to a collective decision.
We learned about two broadly different ways of coming to a decision:
-
Deliberation, where reviewers met in-person to discuss projects, share opinions, and come to a decision together. These tend to be intense, facilitated sessions.
-
Group voting, where reviewers gathered information from applicants through questions or discussion, then cast votes on projects.
Groups doing deliberation also tended to use voting within the group as an informal way of learning each person’s position before coming to a group decision.
“The panel meeting is usually really full on, 3 or 4 hours with 5 minutes break every hour.”
In the prototype we’ve illustrated using the second method, group voting.
Dot voting
Both FundAction and Edge Fund used a scheme where every reviewer had multiple votes to allocate to projects. (This is like “dot voting” where people have 5 dot stickers to add to post-it notes on a wall.) This works well for their use case.
One property of dot voting is that every reviewer must read every application in order to choose how to allocate their votes. They also have to be able to hold all the ideas in their head at once in order to make a decision.
We were interested in the idea of reviewers not having to look at all applications.
Is it possible - given a large enough number of reviewers - to reach a meaningful group decision when each reviewer only votes on 20% of applications? That could make it less onerous to be a reviewer and increase the number of people able to participate.
For more detail on general problems with dot voting, see this blog post: Dotmocracy is broken.
Yes, No, Maybe
We took inspiration from the way voting was used informally in deliberation sessions and applied it to the prototype.
We heard about a scheme where each reviewer looked at each application in turn and made a private Yes, No or Maybe decision on that application.
When everyone had voted on an application, the group came together with a number of Yes, No and Maybe votes. They could then agree on how to turn that into a group vote (for example, “if we all said yes or maybe, call it a yes”)
This type of voting supports our use-case that not every reviewer has to vote on every application. As long as each application gets enough people voting on it, it can be given a score based on the weighting of Yes, No and Maybes. That score can be compared against other projects to find the winners.
It may also be less susceptible to block voting, where where someone asks all their friends to go and vote for a single project.
Recommendations
- Design primarily for mobile.
- Support reviewing in several short sessions. Keep track of progress, make it easy to jump back in.
- Present everything in one place. Don’t make reviewers switch between different documents
- Randomise the order of applications. Each reviewer should see a consistent order, but different from everyone else’s order.
- Ask “feel” questions for voting, for example “do you feel we should fund this project?”. This emphasises that it’s the reviewer’s personal opinion that’s valued in the process.
- Make reviewing less onerous by supporting voting on a portion of applications. By making it a smaller commitment, it should be possible to recruit more reviewers, and particularly not just those with lots of spare time. Use a voting system that can accommodate this, with minimum voting thresholds.
- Support voting in lots of short sessions rather than having to commit to a single time to vote on everything.
- Collect feedback for No votes. Frame these as constructive improvements and automate collating and sending them to the applicant. This gives legitimacy to the process, helps the applicant develop their application and provide essential transparency.
- Show the personal highlights the person made when reviewing the application. This helps jogs their memory to things they thought were important before.
- Show the best questions and answers. Hide the questions which didn’t get many upvotes.
- Show progress as people vote on applications. It’s helpful to know roughly how much time it will take and to know that the task isn’t endless.
- Allow changing votes until the deadline. This reduces the friction of casting votes and supports voting in several short sessions.
- Automate email reminders as deadlines approach. Make it easy to jump back in to the reviewing process.
Prototype
In the second week of the month, Tina receives an email telling her it’s time to vote on ideas.
Tina opens the first idea from the email and jumps straight to a page where she can vote.
She sees the highlights she made when reviewing the application and remembers that she thought it was good value for money.
She sees that a new question has been asked by another member since she last looked at the idea.
The question raises doubts in Tina’s mind. She feels the money shouldn’t be sent to the asylum centre until things have been resolved.
She votes “No” and writes some feedback asking the person to re-apply in a few months.
After re-reading and voting on several more ideas, Tina needs a break. She clicks “See all” to check her progress.
She sees that the deadline is 10pm on Sunday and makes a mental note to have another go when the kids have gone to bed.
Tina had forgotten all about voting until she sees an email from FacilitatorBot.
She understands that a certain number of people need to vote to make it legitimate. She sees there’s not much time so decides to finish voting right away.
She’s motivated by wanting the money to be awarded, and not wanting her previous efforts to have gone to waste.