A few days ago BuzzFeed's Craig Silverman published his annual overview of the top 50 biggest on-line hoaxes of the previous year. Just like last year we used this list as a basis to do some analysis to see how Lead Stories did compared to other fact checking websites. But this year we used the full top 50 instead of just the ten first stories and we included four other fact checkers in the analysis instead of just three.
Of course this analysis has to be taken with a huge grain of salt: not all fact checking websites use the same criteria when it comes to selecting what to check. Some choose to focus on checking the claims made by public or political figures, others might watch what is appearing in the media while others (like Lead Stories and Snopes) may be specifically hunting for the type of junky viral hoaxes that make up the bulk of the BuzzFeed list.
With that being said, how did we stack up against Snopes, Politifact, FactCheck.org and AP when it came to identifying and debunking the hoaxes from this list?
Not too bad, it seems.
Of the fifty top hoaxes identified by BuzzFeed only seven were not debunked by any fact checker (some were just too silly or obviously satirical in nature, for example Stevie Wonder getting arrested for watching kiddie porn...).
The 43 remaining ones were all checked by at least one fact checker:
- 13 were only checked by Lead Stories
- 7 were only checked by Snopes.
- 22 were only checked by Lead Stories and Snopes
- 1 story was only checked by Snopes and AP
- Politifact, AP and FactCheck.org didn't check any of the stories that Snopes or LeadStories hadn't checked first
The final tally was:
- 35 Lead Stories
- 30 Snopes
- 7 AP
- 7 Politifact
- 5 FactCheck.org
For the vast majority of hoaxes covered by Lead Stories we were also first to write about them.
We put our full analysis in a Google Docs spreadsheet which can be viewed here.
Methodology/legend:
- For all stories in the table we marked the fact checks in order of pubilcation with the first three being marked gold, silver and bronze.
- If articles appeared on the same date they all get credit for being first, second or third.
- All articles published by Lead Stories are automatically tweeted within five to ten minutes after publications, so we used the timestamps and links to those tweets as the baseline. All times in CET.
- For Snopes we also looked at the CET date/times of the first tweets about a story (where available) but we took the publication date on the site as the baseline (which was often earlier).
- For the other sites we only looked at the publication dates on the sites.
- For cases where Lead Stories and Snopes published on the same day the first to tweet is marked in bold.
- We searched for tweets by using Twitter search with a query of the form "from:leadstoriescom" or "from:snopes" followed by keywords related to each hoax.
- We searched for articles by using Google with a query of the form "site:apnews.com" followed by keywords related to each hoax.
- Spot a mistake? Do let us know and we'll add a correction.
