How Community Notes Is Revolutionizing Online Fact-Checking

Community Notes is transforming the way online misinformation is addressed through an innovative crowdsourced fact-checking system. This article provides an in-depth look at how Community Notes works, the principles behind its success, and its impact on combating false information online.

What is Community Notes?

Community Notes is a system on X (formerly Twitter) that allows users to add context to posts that may be misleading. Here's how it works:

• If a user sees a potentially misleading post, they can propose a note with additional context or information
• Other users then rate whether they find that note helpful
• Notes that are found helpful by people across different perspectives are then shown publicly on the post

The goal is to provide users with more information about what they're seeing so they can make better informed decisions.

The Algorithm Behind Community Notes

What makes Community Notes uniquely effective is its bridging-based agreement algorithm. Some key aspects:

• It looks for agreement on notes from people who have disagreed in the past on other topics
• This surprising agreement across different perspectives helps ensure notes are neutral, accurate and well-written
• The algorithm provides anti-manipulation properties without needing to rely on centralized fact-checkers

As Keith Coleman, product lead for Community Notes, explains:

"We actually look for agreement from people who have disagreed in the past. And what we see is when people actually have that sort of surprising agreement, that's what makes the notes so neutral and accurate and well-written really overall."

This approach allows Community Notes to bootstrap a reliable fact-checking system without needing external labels or centralized arbiters of truth.

Key Principles Behind Community Notes

Several core principles have been crucial to the success of Community Notes:

Representing the Voice of the People

Community Notes is designed to represent the voice of users, not the company. There is no button for X employees to change the status of notes - what shows is purely determined by user ratings.

Open to All

Anyone can sign up to be a Community Notes contributor (with basic criteria like having a verified phone number). Contributors are randomly selected to participate.

Full Transparency

All of the code and data behind Community Notes is open source and publicly available. This allows for complete auditability.

Focused on Helpfulness

The goal is to show notes that are genuinely helpful to a broad range of people, not just factually correct.

Sticking firmly to these principles, even when it was challenging, has been key to building trust in the system.

Impact and Effectiveness

Community Notes has proven remarkably effective at combating misinformation:

• Hundreds of notes are added per day, spanning topics from politics to entertainment
• Notes are seen billions of times per year
• External studies have found Community Notes changes people's beliefs about claims in posts
• Posts with notes see 30-40% drops in engagement

The system has been particularly impactful during major events like conflicts, where it can rapidly debunk false claims and out-of-context media.

How Community Notes Evolved

The journey to develop Community Notes (originally called Birdwatch) provides interesting lessons in product development:

Origins

Keith Coleman initiated the project in 2016 after joining Twitter, seeing the critical need to address misinformation at internet scale.

Iterative Development

The team took an extremely iterative approach, constantly testing assumptions:

• Started with mockups to gauge interest
• Ran small pilots to test if regular people could write good notes
• Gradually expanded while closely monitoring quality

Organizational Structure

Community Notes was set up as a "thermal" project within Twitter - a small, focused team with high autonomy. Key aspects:

• One clear decision maker (Keith)
• 100% team focus
• Freedom from standard processes like OKRs
• ~5 person cross-functional team

This structure allowed for rapid iteration and decision making.

Surviving Leadership Changes

Remarkably, Community Notes has survived multiple CEO changes at Twitter/X. Keys to this:

• The product's inherent value in producing helpful information
• Consistent execution and progress from the team
• Focus on solving a real, important problem

Technical Implementation

Some interesting technical aspects of Community Notes:

• Uses matrix factorization trained with gradient descent to determine helpfulness scores
• Architected to be fully open source and replicable, even if not optimal for production
• Runs on a single machine with 500GB of RAM

Future Directions

Some potential future developments for Community Notes:

• Leveraging AI/LLMs to assist note writers
• "Supernotes" - using LLMs to generate and simulate ratings for many note variants
• Further improving speed and coverage of notes
• Potentially having the core algorithm itself be community-developed

Broader Implications

Beyond its direct impact, Community Notes demonstrates some important things about society:

• There is more agreement across polarized groups than we often assume
• Given the right systems, people are eager to contribute to the common good
• Transparency and community involvement can build trust, even on contentious topics

As Keith Coleman notes:

"Community Notes is cool itself, but I think what it points to about society is actually even bigger. Society often feels really polarized...but actually Community Notes shows you people really can agree on quite a lot, even on super controversial topics."

This provides reason for optimism about our ability to find common ground and address major challenges as a society.

Key Takeaways for Product Development

The Community Notes story offers valuable lessons for product development:

Small, Focused Teams Can Have Huge Impact

A small team of 5-6 people built a system now used by millions and adopted by other major platforms.

Iterative Development with Real-World Testing

Constantly testing assumptions and iterating based on real usage was crucial.

Stick to Core Principles

Unwavering commitment to principles like transparency and community ownership was key to building trust.

Let Users Guide Development

Many valuable use cases emerged organically from how users chose to use the product.

Optimize for Speed and Autonomy

The team's ability to make rapid decisions and ship quickly was a major advantage.

Conclusion

Community Notes represents a breakthrough in addressing online misinformation. By leveraging the wisdom of the crowd in a carefully designed system, it has created a scalable, trusted way to add context to potentially misleading information.

Its success demonstrates the power of community-driven approaches and offers a model for how we might tackle other major societal challenges. As the system continues to evolve and expand, it has the potential to significantly reshape how we consume and verify information in the digital age.

For product builders, Community Notes offers inspiring lessons in the impact that small, focused teams can have when empowered with autonomy and guided by clear principles. It's a testament to the potential of innovative product development to address critical real-world problems.

Article created from: https://www.youtube.com/watch?v=8dgyqYHLcCI&t=3s