WISDOM Readme and Github repo

We’re developing a decentralised Open Science framework for recognising and rewarding contributions to the commons, so that we can empower contributors and connect diverse communities with aligned goals.

Recognition and reward systems often fall short, however, failing to capture the full range of contributions, recognise their unique qualities, or reward contributors fairly. More broadly, the incompatibility of different systems means that aligned communities often fail to cooperate, where they could otherwise be collaborating to achieve their mutual goals.

 

The WISDOM Solution

WISDOM is a flexible framework designed to crack the code of cultural change. By creating a standardised, universal measure of gratitude for contributions, WISDOM aims to empower contributors and connect aligned communities in the pursuit of their common goals.

At the heart of WISDOM is an inclusive review protocol that incentivises feedback about diverse contributions. This input is then converted into a rich information matrix, recognising the unique qualities of every recorded contribution. Contributors can then be rewarded with tokens representing the collective gratitude for their contibutions, which can be paid forward in thanks to other people in turn. Contributor expertise and reliability can also be expressed numerically, creating a platform for people who consistently deliver value to their community.

By integrating the three key functions of information sharing, economics and governance, WISDOM aims to become a complete ‘community operating system’ for literally any community.

 

Minimal Viable Model (MVM)

0. Embed Values. Every community can embed their values directly into the code in the form of dimensions that reviewers vote on. WISDOM builds on advanced open evaluation models that propose a common ‘primary dimension’ to facilitate cross-comparison between communities. Here, we propose “Gratitude” as the primary dimension because of its universal applicability and psychological benefits for communities of practice. Beyond Gratitude, however, each community can define as many dimensions as they wish and evolve them in line with feedback and metaresearch on the data.

 

1. Record. Contributions are recorded in a transparent register of the community’s choice. The scale and scope of records depends on the community, and members are incentivised to keep the register current, complete and accurate.

 

2. Review. WISDOM uses pairwise comparisons to generate ratings at the crowd-level. Pairwise comparisons are a quick, simple and user-friendly review protocol, ensuring that diverse people can contribute data, regardless of expertise or technical prowess. Anyone in the community can complete a pairwise comparison by voting between two contributions on each dimension of interest (e.g., “Which of these two contributions are you most grateful for?”). Reliable reviewers can also earn the status of ‘meta-reviewer‘, which entitles them to vote on the relative value of reviews themselves (termed a ‘meta-review‘).

 

3. Recognise. Pairwise comparisons are relatively uniform, meaning that we can treat them as a ‘standard unit’ of contribution and use them to scale all other contributions. We do this by including a range of review contributions (e.g., 32, 64, and 128 pairwise comparisons) in the meta-review stage and fitting a function to the resulting votes, then interpolating values for all non-review contributions (see 3. Recognise tab in our pilot dataset). The result is a numerical value representing the collective gratitude for every single contribution (see Gratitude per contribution chart in our pilot dataset). Communities who include more than one review dimension can generate a multi-dimensional information matrix about each and every contribution (see Contribution Qualities chart in our pilot dataset), which is infinitely filterable and ideal as a high-signal training ground for AI models.

 

4. Reward. Contributors are rewarded with tokens representing the collective gratitude for their contributions (see OHMnoms per Contributor chart in our pilot dataset). Reviews are rewarded at the rate of 1 token per pairwise comparison (including all dimensions), and regular contributions are rewarded commensurate to the number of units on the Gratitude dimension (more complicated transfer functions could be explored in the future). Communities may agree to subtract common costs (e.g., the average cost of an event), leaving a balance that reflects each contributor’s total energetic exchange with the community (see OHMnom Balances chart in our pilot dataset). Excess tokens can be paid forward to other people who the contributor wishes to thank in turn, forming the basis of an evidence-based gifting economy.

 

5. Respect. By measuring contributions over time, we can generate metrics reflecting reviewer reliability and expertise within the community. Reviewer reliability can be measured using a range of measures (e.g., test-retest reliability, internal consistency), and trustworthy reviewers can be elevated to the status of meta-reviewer (see Reviewer Reliability chart in our pilot dataset). Experts in particular fields and topics can be identified by tracking the value they add to that field over time (e.g., see Expertise: Science chart in our pilot dataset). Collectively, these metrics could form the basis of a merit-based participatory democracy algorithm, in which votes are weighted according to each person’s validated expertise in the topic of interest.

 

6. Replicate and Repeat. All of the above processes should be conducted transparently and shared with the community, so that everyone can learn and grow together. The final stage is therefore to replicate the previous materials, embed any learnings, and repeat the process again.

 

Principles and Features

• Transparent. Contributions and reviews are shared publicly, so you know what’s going on behind the scenes.
• Inclusive. Everyone is rewarded for their unique contributions, including reviews, meaning that everyone can contribute in the way that works for them.
• Replicable. Open source and forkable, so you can stand on the shoulders of giants.
• Accessible. Our review protocol is user-friendly and cognitively simple, so that anyone can contribute meaningful data no matter their circumstances.
• Collaborative. Our crowd-based reward mechanism incentivises collaboration over competition.
• Diverse. Our framework is flexible, scaleable and can be customised to the needs of different communities.
• Honest. Transparent reviews and reliability metrics encourage honesty and guard against cheats.

 

A Universal Framework

WISDOM is designed to be maximally inclusive to all communities. The MVM above is the simplest model possible for capturing quantitative meta-information — information about the relative value of information. There are of course more nuanced ways to conduct reviews, but all of these can be considered derivatives of the MVM we outline here. Each community could develop additional modules that plug into this core model (e.g., a qualitative review module), using the pairwise comparison method to determine the relative value of contributions to those layers (e.g., the value of a qualitative review could be given in terms of an equivalent number of pairwise comparisons).

Communities are also free to develop their own terms, standards, processes and algorithms. For example, at OHM, we call our tokens OHMnoms, but other communities might want to name their own tokens accordingly. Each community could then decide what other communities to cooperate and trade with, by assessing their transparent processes and data and deciding (if necessary) a relative weighting for the respective tokens. Providing that all communities adopt the MVM we present here, it would create a fundamental communication layer both within and between communities that directly encodes gratitude and facilitates cooperation toward common goals.

 

Usage Examples

1. Gatherings. See data and results from a Open Heart + Mind (OHM) gathering called Tiny OHM (see also /reports). We’re also prototyping with Vibe Camp to review contributions to Vibeclipse.
2. Conferences. See our working prototype to evaluate contributions to the 2023 AIMOS open/meta-science conference.
3. Scholarly communication. Our model was originally designed to solve cultural inertia in the transition to open science platforms. If you work in the scholarly communication / evaluation space, we’d love to hear from you!
4. Literally any contribution to any community. We think of WISDOM as a minimal universal model, meaning that it is both flexible and adaptable to different community needs. We’re collecting a list of organisations interested in exploring the framework, please add your name here if interested so we can get in touch.

 

Development History

• 2019-2021: WISDOM designed as a solution for cultural inertia in academia
• Nov 2021: Hosted Heart + Mind Festival, used to prototype WISDOM v0 (token allocation)
• Jan 2022: Open Heart + Mind (OHM) founded to prototype the model in the safer context of gift-based gatherings
• Feb 2022: Applications to OHM Gathering open, used to prototype WISDOM v1 (structured rubric)
• Mar 2022: Pairwise comparison model proposed to community (WISDOM v2)
• May 2022: Development of WisdOHM app begins (note this app will be open sourced as soon as we can resolve some sensitive data issues in the codebase)
• June 2022: Hosted Tiny OHM, used to prototype WISDOM v2.0 / v2.1 and collect first complete dataset
• June 2023: Hosted OHM Gathering (data collection waiting on app)
• Nov 2023: Presentation at AIMOS conference; AIMOS collaboration begins
• Apr 2024: Workshop and Hackathon at Vibeclipse; Vibecamp collaboration begins

Note that our first two years of development were recorded in our transparent Clickup workspace (contact Cooper Smout for access). In March 2024 we began migrating to Github (work in progress).

 

Roadmap (work in progress)

• Apr 2024: Networking tour of North America and Colombia
• June 2024: Presenting at SIPS conference (Kenya)

 

About the Name

WISDOM stands for Weighted Information Schema for Distributed Open Merit. The name also refers to the Wisdom of the Crowd effect: the idea that large groups of people are collectively smarter than individual experts when it comes to problem-solving, decision-making, innovating, and predicting.