Is fact-checking on Facebook done by the Australian data company Appen? No, that's not true: Fact-checking on Facebook is done in the U.S. by 10 members of the independent International Fact-Checking Network (IFCN), a worldwide forum whose members are certified to adhere to a Code of Principles, one of whose tenets is non-partisanship. Appen, which has a contract with Facebook to provide lower-level "community reviewers," is not a member of the IFCN. The "whistleblower," if he or she worked for Appen, was most likely a part-time freelance contractor who was not involved in actual fact-checking and who has misrepresented the fact-checking process and criteria used by Facebook.
The claim appeared in an article (archived here) published by Jean-Pierre "Raven" Gregoire on LinkedIn on November 22, 2020 under the title "Facebook 'Fact Checkers' Trained to Censor Conservatives Exclusively". It opened:
The message below was sent to me by a friend who is a former Facebook Fact Checker.
I can't say this on Facebook, but I feel it's important to say. I was a Facebook Fact Checker. And your conspiracy theories about Facebook, are more true than you realize.
This is what the post looked like on LinkedIn at the time of writing:
(Source: LinkedIn screenshot Fri. Dec. 18, 17:35 UTC 2020)
The post continues:
I worked for a company named Appen. We are a 3rd party freelance contract company. A few years ago Facebook approached us with an offer. Zuckerburg couldn't legally censor people on his platform because he told congress that he was an open forum. So, he uses several 3rd party companies to "Fact Check", and otherwise censor, information.
Before detailing how fact-checking is done on Facebook and what criteria are involved, it is first necessary to set the record straight on the false allegations in the post by the "whistleblower."
Foremost: the person who claims to have worked for Appen and wrote the story would not have been a fact-checker. Facebook spokeswoman Andrea Vallone told Lead Stories in a December 14, 2020 email:
I can confirm Appen is not one of our independent fact-checking partners (global list here).
Instead, Facebook hired Appen in late 2019 to provide a stable of "community reviewers." Vallone detailed the difference between community reviewers and fact-checkers in another December 14, 2020 email to Lead Stories (italic emphasis hers):
Research from community reviewers is meant to complement the efforts of professional fact-checkers, and differs from the content validation process that these fact-checkers follow. Fact-checking requires original reporting that entails conducting research to confirm data points, verify photos and videos, or cross-check transcripts in order to debunk or support journalism. In contrast, these community reviewers can give directional signal to professional fact-checkers that content may have a higher likelihood of being false if people from the community cannot find evidence to corroborate or refute a claim.
How it works (bold face by Vallone):
Content is reviewed by multiple community reviewers, the results of whose collective research get surfaced to fact-checkers. Reviewers will be audited on an ongoing basis to ensure they're adhering the guidelines and training that outlines the process. Ultimately, professional fact-checkers must still review and rate posts, and only upon false rating from a fact-checker will content see reduced distribution in News Feed.
Additional information on the community reviewers can be found in this Facebook announcement and this story by the Poynter Institute, which organized the International Fact-Checking Network (IFCN) in September 2015.
As to Appen, the Facebook spokeswoman explains:
We have partnered with Appen, an outside vendor, who sources, vets, selects, and qualifies community reviewers, who are part-time contractors. Appen manages this process with protections in place to avoid gaming and manipulation. Appen recruits potential new community reviewers and selects them based on several demographic and behavioral criteria. Global public opinion and data company, YouGov, conducted an independent study of community reviewers and Facebook users and determined that these recruitment requirements yielded reviewers that are representative of the Facebook community in the US and reflect the diverse behaviors, demographics, and viewpoints -- including political ideology -- of Facebook users.
Appen did not respond to written requests by Lead Stories for comment on December 14, 2020 and December 15, 2020.
The "whistleblower's" allegations were posted on LinkedIn by Jean-Pierre "Raven" Gregoire, who identifies himself on the site as an "inspirational author, spiritual teacher, content creator, life coach, lover of life, ghostwriter, copy editor." It is not clear if and how Gregoire knows the person whose false allegations about Facebook fact-checking he published. Gregoire did not answer a written request through Facebook Messenger for that information on December 17, 2020.
The message Gregoire posted makes a series of allegations, most of which are not true or incorrect. Among them:
Claim: Conservative and right-leaning posts on Twitter and Facebook make it to the Appen service, and left-leaning posts are to be ignored...
Again, Appen freelance "community reviewers" do not write fact-checks. Although Facebook does refer content to community reviewers, political or ideological orientation is not a criteria. In a December 17, 2019 posting announcing the creation of the community reviewer program, Facebook outlined the criteria for sending content to the Appen contractors:
Our machine learning model identifies potential misinformation using a variety of signals. These include comments on the post that express disbelief, and whether a post is being shared by a Page that has spread misinformation in the past. If there is an indication that a post may be misinformation, it will be sent to a diverse group of community reviewers.
Claim: Facebook created a program that feeds automatically into Appen's system.
As noted above, Facebook routes content to community reviewers, not to Appen.
Claim: Appen flags an entire article as disputed/false/discredited/untrue even if there's only one idea that's not completely confirmed.
Appen community reviewers are supposed to flag content for possible further investigation by fact-checkers, and all it takes is one wrong or untrue statement for the entire article to be referred onward. Poynter points out in a December 17, 2020 article that community reviewers "will be asked to identify the main claim in each post. After that, they will have to find sources that can either support or refute that claim, to conclude whether a post is corroborated or not."
But the community reviewers perform a preliminary function. Says Facebook in its announcement: "They are not making final decisions themselves."
Claim: Community reviewers write articles for Politico, The New York Times and other publications and can "mark a title or article as true as long as at least three of the core ideas in the article or post are potentially true."
Community reviewers do not write articles for any plublications as part of their duties for Appen. They provide evaluations for Facebook. Any article a community reviewer writes for an outside publication would be done in an individual, private capacity. If any content contains any untruths or errors, it cannot be labeled as true. It has to be 100 percent true to earn that designation.
Claim: When community reviewers write articles, they can cite their own articles as evidence.
This is vague and unclear. Write evaluations of content for Facebook? Write, as Appen contractors, for outside sources, which it has been established they cannot do? And which "own" articles would they be citing?
Claim: Appen only hires community reviewers if they pass ... an opinion survey, and "they must answer all left leaning."
Facebook does vet its hires for the community reviewer pool, but there's no ideological litmus test, and certainly no bias toward left-leaning candidates. As noted by Vallone above and also mentioned in the platform's announcement of the program:
To ensure the pool of community reviewers represents the diversity of people on Facebook, we're partnering with YouGov, a global public opinion and data company. YouGov conducted an independent study of community reviewers and Facebook users. They determined that the requirements used to select community reviewers led to a pool that is representative of the Facebook community in the US and reflects the diverse viewpoints -- including political ideology -- of Facebook users. They also found the judgments of corroborating claims by community reviewers were consistent with what most people using Facebook would conclude.
Facebook has distilled its efforts against untruths and inaccuracy to one sentence:
The focus of this fact-checking program is identifying and addressing viral misinformation, particularly clear hoaxes that have no basis in fact.
According to Facebook, this is how it's done:
Fact-checking partners prioritize provably false claims, especially those that are timely or trending and important to the average person.
Fact-checkers can review and rate public Facebook and Instagram posts, including ads, articles, photos, videos and text-only posts.
Fact-checking partners do not rate content that does not include a verifiable claim, or content that was true at the time of writing. In addition, opinion and speech from politicians is not eligible to be fact-checked.
Fact-checkers pick content to review based on each organization's criteria. Lead Stories details on its website how it finds claims and stories to investigate and what tools it uses.
Facebook lists its five steps in fact-checking and the potentially resulting actions:
Note that not all of these actions are applied automatically all of the time, media reports indicate sometimes exceptions are made.
Facebook's rating system for content examined by fact-checkers also is fairly detailed and nuanced:
- False: Content that has no basis in fact, including fake quotes, untrue claims, unfounded conspiracy theories with implausible conclusions, fabricated content and misleading images, audio or video.
- Altered: Image, audio or video content that has been edited or synthesized beyond adjustments for clarity or quality, in ways that could mislead people.
- Partly false: Content that has some factual inaccuracies, including a mix of true and false claims. Also content presented as an opinion but based on underlying false information and inaccuracies or miscalculations about numbers, dates and times.
- Missing context: Content that may mislead without additional context or explanation.
- Satire: Content that uses irony, exaggeration or absurdity for criticism or awareness, particularly in the context of political, religious or social issues, but that a reasonable user would not immediately understand to be satirical.
- True: Content that contains no inaccurate or misleading information.
Publishers that have had their content rated can also appeal or issue a correction under the rules of the fact checking program, in which case the rating (and it's associtated effects) can be lifted or changed.
As stated earlier, verified fact-checkers belong to the International Fact-Checking Network. Lead Stories and the other nine U.S. organizations in the IFCN have agreed to a Code of Principles, which is "a series of commitments organizations abide by to promote excellence in fact-checking." The IFCN defines it this way:
This code of principles is for organizations that regularly publish nonpartisan reports on the accuracy of statements by public figures, major institutions, and other widely circulated claims of interest to society. It is the result of consultations among fact-checkers from around the world and offers conscientious practitioners principles to aspire to in their everyday work.
Primary among the principles is "a commitment to non-partisanship and fairness:"
Signatory organizations fact-check claims using the same standard for every fact check. They do not concentrate their fact-checking on any one side. They follow the same process for every fact check and let the evidence dictate the conclusions. Signatories do not advocate or take policy positions on the issues they fact-check.
The other four principles are commitments to:
- standards and transparency of sources.
- transparency of funding and organization.
- standards and transparency of methodology.
- an open and honest corrections policy.
The IFCN has strict requirements for membership:
Applications are assessed by independent assessors for compliance with 31 criteria. Their assessment is reviewed by the IFCN advisory board to ensure fairness and consistency across the network.
In addition:
IFCN signatory status may be granted to legally registered organizations set up for the purpose of fact-checking that regularly publish non-partisan reports on the factual accuracy of statements by public figures and prominent institutions and widely circulated claims in text, visual and other formats focused primarily on claims related to public interest issues.
IFCN signatory status may not be granted to organizations whose editorial work is controlled by the state, a political party or politician. It may however be granted to organizations that receive funding from state or political sources to carry out public service journalism if the IFCN assessor determines there is clear and unambiguous separation of editorial control from state or political influence.
In addition to Lead Stories, the other nine U.S.-organizations in the IFCN are:
- AFP
- Check Your Fact
- FactCheck.org
- MediaWise
- Reuters
- PolitiFact
- The Dispatch
- The Washington Post Fact Checker
- USA Today
Facebook picks which partners it wants to work with for which countries, and not all members of the IFCN want to perform fact-checking for the social media platform. The Washington Post is not part of the program for example, and Science Feedback (though originally registered in France) also fact checks for Facebook in the United States.
Following IFCN members are currently part of the program for the United States:
- AFP
- The Associated Press
- Check Your Fact
- The Dispatch
- FactCheck.org
- Lead Stories
- PolitiFact
- Science Feedback
- Reuters Fact Check
- USA Today
Facebook also relies on more than 85 global third-party fact-checking organizations.
Fact-checking is different from enforcing Community Standards, which is a function Facebook performs. The platform's community standards are a guide for what is and isn't allowed on Facebook and include:
Authenticity: Making sure content is authentic and that people don't use Facebook to misrepresent who they are or what they're doing.
Safety: Expression that threatens people can intimidate, exclude or silence others and isn't allowed.
Privacy: Protecting personal privacy and information.
Dignity: All people are equal in dignity and rights, and people should respect the dignity of others and not harass or degrade others.
Facebook notes that the consequences for violating the platform's Community Standards vary depending on the severity of the violation and the person's history on the platform. For example, a first violation may warrant only a warning, but continued violation of the policies could result in a restriction on the person's ability to post on Facebook or having his or her profile disabled. Facebook also could notify law enforcement officials when the platform believes there is a genuine risk of physical harm or a direct threat to public safety.
Editors' Note: Facebook is a client of Lead Stories, which is a third-party fact checker for the social media platform. On our About page, you will find the following information:
Since February 2019 we are actively part of Facebook's partnership with third party fact checkers. Under the terms of this partnership we get access to listings of content that has been flagged as potentially false by Facebook's systems or its users and we can decide independently if we want to fact check it or not. In addition to this we can enter our fact checks into a tool provided by Facebook and Facebook then uses our data to help slow down the spread of false information on its platform. Facebook pays us to perform this service for them but they have no say or influence over what we fact check or what our conclusions are, nor do they want to.
