NewsTrust, a non-profit startup aimed at improving the credibility of media, spent a week earlier this year on an experiment it called Truthsquad — a project that tried to “crowdsource” fact-checking, and specifically some of the major statements made by public figures and special-interest groups about issues such as health care. NewsTrust founder Fabrice Florin has released some of the results from that experiment, saying the project was a success and that NewsTrust and its partners plan to expand the effort over the next year.
Among the things that NewsTrust learned, according to Florin:
- Game mechanics work: NewsTrust says the Truthsquad experiment generated twice as much participation as some of its other pilots in 2010. Users gave ten times more answers per quote than reviews per story, over half the participants read linked stories, and a third answered a Truthsquad quote. Florin said the high level of engagement was partly due to the game-like quality of the user experience, which started by inviting people to guess whether a statement was true or false.
- Pros and amateurs can work together: Truthsquad was a collaboration with advisors and journalists from the Poynter Institute and FactCheck.org, and Florin said that having experienced professionals judge results and help users with the fact-checking avoided “some of the pitfalls of pure crowdsourcing initiatives, which can turn into mob scenes — particularly around politically charged issues.” Amateurs learned valuable skills, he said, and some community members posted links that were “critical to reaching our verdicts.”
- Crowdsourcing takes effort: Florin said that despite high levels of participation, Truthsquad “didn’t get as many useful links and reviews from our community as we had hoped,” and as a result, the startup’s editorial team had to do a lot of work researching the evidence behind the statements they were checking, making the whole effort more labor-intensive than the group expected. Florin said that in the future, NewsTrust is going to experiment with more visible rewards for input, such as badges, redeemable credits of some kind and possibly even prizes.
Truthsquad isn’t the only attempt to bring a crowdsourced element to media fact-checking: MediaBugs, founded by former Salon editor Scott Rosenberg, is another fact-checking oriented site that is funded by a Knight News Challenge grant and recently announced it has moved out of beta and is going national. The startup has a widget that allows publishers to integrate crowd or reader-driven fact-checking into their websites, and is hoping to find other ways to distribute its services, rather than making people go to the MediaBugs site.
Like most blog-focused sites, we get plenty of fact-checking from our readers, through the comments on our posts — and it is a critical part of what we do (unfortunately, we are not perfect). Many traditional publishers, however, don’t get as much as they probably need, and both MediaBugs and NewsTrust are interesting efforts in that direction.
NewsTrust, meanwhile, says it is hoping to fact-check one public statement a day for the next year using Truthsquad. Florin says he also wants to help train users and readers in how to fact-check statements by politicians and others in the media, using the skills of the startup’s advisors, including the Poynter Institute and FactCheck.org. That’s something that we could probably all benefit from. If you want to participate, you can sign-up here.