Draft 2 - ILI Digital Credentialing: Evaluate Sources

Primary, secondary and tertiary sources

One of the factors you must consider when evaluating a source that you have found is its value as a "primary,"  "secondary," or even "tertiary" source. What do these terms mean, why are they important, and when would you use each of these types of sources?

Primary sources

Primary sources are your evidence to support your claims, thesis or hypothesis. A historian might claim, "During World War II, private citizens were encouraged to demonstrate their patriotism by cultivating Victory Gardens." A political scientist might claim, "Early voting opportunities positively impact voter turn-out." A scientist might hypothesize, "Micro-plastics negatively impact human health by increasing inflammatory response in key organs." If they make these claims, critical thinkers expect them to provide evidence to back them up. The historian would need, perhaps, a government pamphlet, distributed during World War II, that linked patriotism and Victory Gardens. The political scientist would need statistics comparing voter turn-out in a region when early voting is and is not available. The scientist would need data gathered from an experiment demonstrating how micro-plastics impact the inflammatory response. Pamphlets, statistics gathered through surveys or observation, and experimental data are all types of primary sources for various disciplines. Here are some specifics about primary sources in various disciplines:

Identifying useful primary sources is critical in research. Watch this video, where experts from Centre College discuss how they engage with primary sources:

Secondary sources

Secondary sources interpret, assign value to, conjecture upon, and draw conclusions about the events or results reported in primary sources. Most scholarly articles, including the academic papers students write, where authorities make a claim and support it using evidence from primary sources, are secondary sources. Many articles published in blogs, newspapers, and magazines, where the author makes an assertion and supports it with evidence, are secondary sources. When a researcher writes a paper, they include references to secondary sources in order to synthesize what is already known on their topic - the conclusions prior researchers have drawn - to set the stage for the new information they intend to contribute. (This is usually done in the "Literature Review" section of a scholarly paper). Here are some specifics about secondary sources in various disciplines:

Watch this very brief video for more information about a particularly valuable secondary source: review articles.

Tertiary sources

Tertiary sources summarize the information in primary and secondary sources to provide background on a topic, idea, or event. They are typically used for quick reference and exploration - to quickly learn the important facts about a topic. Examples of tertiary sources include: encyclopedias, dictionaries, almanacs, fact books, directories, guidebooks, handbooks, manuals, and indices.

Librarian TIP: Sources become primary rather than always exist as primary sources. What does that mean? For example, if a person in December 2022 reads a newspaper article interpreting and analyzing the 2022 US election, that is a secondary source from their perspective. To a historian in the year 2200, that newspaper article will be a valuable primary source, providing evidence of a certain historical point of view/attitude.

Popular sources, trade journals, and scholarly / peer-reviewed sources

When doing research and contributing to the scholarly conversation, you must be concerned about the credibility of the sources you use. Your own credibility depends upon the judgment you demonstrate while choosing sources to support your arguments. You will often hear people arbitrarily stating that certain sources, like ones found online, are not credible. It is not that simple! Credibility is contextual. The credibility of a source is impacted by:

• how a source of information was produced (how much effort was put into its creation)
• the expertise of the person creating it
• the evidence used to support the assertions made
• the audience the source was created for
• how the researcher uses the source during their research

Let's look at these factors in the different types of secondary sources you will encounter.

Popular sources

Popular articles are typically published in magazines that are for sale in bookstores or supermarket newsstands or, when online, on webpages, blogs or social media. Their intended audience is the general public, with little or no specialized knowledge on the topic. They are written and published quickly, for the purpose of generating sales/revenue, by people with little or no expertise or credentials, who have done minimal research on the topic. They rarely contain specialized vocabulary or intent to empirically prove their claims. They are usually not suitable sources for scholarly research, but depending on the context of their use, they might be valuable. They are often useful for exploratory research, since they are easy to read and help the researcher learn basic information quickly. In this instance, it is unlikely they would be cited as secondary sources in an academic paper. Popular sources might also be valuable primary source evidence of social trends. In that case, they would be cited in academic research.

Trade journals

Materials in trade journals are written by people who are experienced (often very experienced) in their field of employment. They are written for other members of that profession. They often contain specialized vocabulary, appropriate for members of the profession in question. But these articles are written and published quickly in magazines that are sold in popular outlets. They are often not peer-reviewed (see below) and their authors normally do not empirically prove their statements with experimental evidence, but rather with anecdotal evidence, like personal experience. Like popular sources, trade articles are usually not suitable sources to cite as secondary sources for scholarly research. They may be very valuable while exploring a topic for general understanding and especially to learn specialized language and terms that might later be used as keywords when searching databases.

Scholarly materials

Scholarly materials are written by experts with credentials in their field, typically Ph.D.'s, J.D.'s or M.D.'s. You may also hear scholarly materials called academic, refereed or peer reviewed. Many people use these terms interchangeably. The content and claims in scholarly materials are always supported by primary sources, such as experiments (in the sciences) or textual analysis (in the humanities). They also often contain detailed analysis of related, previously published secondary sources. These primary and secondary sources are cited in footnotes and a bibliography. Scholarly materials are written slowly and require much research. Scholarly materials may be published in book or article form. Articles are normally published in a journal, which is typically only available in universities and libraries.

Peer-reviewed articles (or refereed articles) are a specific type of scholarly article that has been evaluated and judged meritorious by other experts in the field. The author of the article submits it for publication and the experts read and evaluate its methodology, analysis and conclusions. Peer review is the most reliable form of scholarship. Journals that contain articles that are evaluated by a board of expert editors are called peer-reviewed journals.

Librarian tip:  Understand that what constitutes authoritative content depends on your information need, not the publication format. Authoritative content may be packaged formally (ex.  library collections, newspapers, textbooks, scholarly journals, databases, reports, patents/standards, codes of practice, government publications. FROM Government agencies, Congress, Research institutes and societies, community and regional organizations, scholars and researchers) or informally (ex. blogs, opinions social Media, letters, interviews, emails. FROM non-experts). Both formal and informal content may be authoritative, depending on how you intend to use it.

Information bias, misinformation, disinformation and mal-information

All sources contain some bias. Learning to recognize and manage bias while conducting scholarship is a critical skill. Popular sources, like social media or articles in magazines or blogs, are most often criticized for containing bias and they are rarely acceptable secondary sources for scholarly research. Why? Because they:

• usually do not rely on empirical evidence;
• are often created in order to generate hits/sales;
• often reflect the lack of expertise and biases of their author or intended audience;
• often contain misinformation or disinformation

It is critical to recognize bias and avoid misinformation when writing credible scholarship. How does bias appear in popular information sources and why is information bias, misinformation and disinformation dangerous? Let's examine this problem briefly by defining some terms:

Misinformation: False, often biased, information, regardless of whether or not it is intended to mislead, deceive or harm people.

Disinformation: False, often biased, information that is spread with the specific intent of misleading, deceiving or harming people.

Mal-information: information that is based on reality, but it is misrepresented and intentionally used to inflict harm.

Bias: An often unreasoned judgment for or against one thing, person, or group compared with another.

Misinformation is often created and disseminated by people who are not well-informed on a topic. A well-known example of misinformation was the shark-eating-hawk:

This still image taken from a video, (shot by a beech-goer and shared on social media) made its way onto many news sites, including FOX and the New York Post. It was billed as the "Bird of prey vs. shark" video. Despite what was reported, this is not an enormous hawk carrying a 12 ft shark. It’s probably an osprey carrying a ladyfish, but the perspective is deceiving. This example of misinformation is relatively harmless - though if enough people took it seriously and stopped going to the beach where this "monster" bird of prey was supposedly hunting, reduced patronage could certainly negatively impact the local businesses.

Disinformation, which intends to deceive, is often much more damaging. Here is a famous example:

TIME magazine apparently editing their photograph of O.J. Simpson to make his skin appear darker was highly controversial when it happened. Now, we know that this type of photo manipulation happens with disturbing regularity. The implication about BIPOC individuals is false, but the intent behind this type of disinformation is to harm and is motivated by bias.

Mal-information is more complicated and is also influenced by personal biases. An example was the publication of the fact that the FBI was investigating Hillary Clinton for her use of her personal email server for official communications while serving as the US Secretary of State. It is a fact that she did use her personal email server in that capacity. Some people would argue that the FBI investigation news was publicized at the time that it was (a few days before the 2016 presidential election) and the way it was (implying her actions were illegal) in order to interfere with Clinton's candidacy in the 2016 presidential election. From that point of view, this news story is mal-information.

The role of bias in misinformation, disinformation and mal-information work

The reasons why misinformation, disinformation and mal-information have become so widespread and impactful are very complex. In simplified terms, this phenomena arises from:

• the collapse of "traditional" news media sources and the corresponding de-emphasis of journalists interrogating the facts in a rush to publish first in the new digital media age
• the trend that news is published on social media, causing journalists to pursue viral stories as opposed to accurate ones
• the emerging news ecosystem which encourages followers of an echo-chamber rather than balanced reporting
• individual bias of news/information consumers

In order avoid misinformation, disinformation and mal-information, information consumers should:

• educate themselves on the strategies used by producers of misinformation, disinformation and mal-information
• fact-check information found all online
• be aware of the types of bias that facilitate the success of misinformation, disinformation and mal-information

We will discuss specific strategies for identifying misinformation, disinformation and mal-information and fact-checking in a later section of this credential. First, let's explore the types of bias we should be aware of and avoid in ourselves.

What types of bias should we be aware of?

	Implicit bias: Bias that results from the tendency to process information based on unconscious associations and feelings, even when these are contrary to one's conscious or declared beliefs. Example: In the USA, if someone says, "Peanut butter and...?" most people will complete that sentence with the word "jelly" because most Americans were exposed to peanut butter and jelly sandwiches as children. That peanut butter and jelly go together is a learned, relatively innocuous, unconscious association. When unconscious bias is associated with attitudes towards groups of humans, it is much less innocuous.
	Blind spot bias: Bias that results from the tendency of an individual to fail to recognize that they may have biases. Example: Since 'bias' is a very charged word often associated with prejudice, most people don't even want to think they may have biases, so they refuse to examine any even innocent preconceptions they have learned through their cultural exposure.
	Confirmation bias: Bias that results from the tendency to look for information or information sources that support your existing beliefs and reject data that goes against what you believe. Example: When a person with left-leaning political views only reads news articles from left-leaning news sources, they are subject to confirmation bias. In research, when a researcher begins with a conclusion in mind and only seeks out evidence to support that conclusion.
	Anchor bias: Bias that results from the tendency to base one's conclusions on information gained early on in the decision-making process. Example: When a person reads only a headline, but not the whole article or the caption of a tweet/image, but does not examine the tweeted link or image more closely. In research, when a researcher assumes from the title or abstract of an article that a certain conclusion is proven in the article. Reading more deeply might reveal contradictions.
	Conservatism bias: Bias that results from an unwillingness to revise one's beliefs when presented with new/different information. Example: A person who continues to believe in a conspiracy theory even after it has been disproven with evidence. In research, when a researcher ignores evidence that contradicts the conclusion they wish to prove.
	Normalcy bias: Bias that results from the belief that the future will closely resemble the past because the past is "normal." Example: A person who believes that the economic situation they grew up with should never change throughout their life, despite the fact that it is normal for the economy to fluctuate. People's belief that COVID would quickly go away and schools and restaurants would open back up in a week or two.
	Halo/Horn effect: Bias that results from one's tendency to believe people they have a positive opinion of and disbelieve people they have a unfavorable opinion of. Example: A person loves a television personality and therefore believes any product they endorse must be worthwhile. In research, over-dependence on well known secondary sources and failure to fully explore.
	Bandwagon effect: Bias that results from the tendency to believe something because a lot of other people do. Example: Giving more credence to something that has been re-posted or thumbed-up many times. In research, over-dependence on journal impact metrics.
	Exposure bias: Bias that results from the tendency for repeated exposure to enhance perception of something. Example: When you repeatedly see the same commercial, you become more and more likely to favor the product its is selling.

How can we counteract bias? 

Some strategies to counteract bias include:

• Critically read information sources using a set criteria, like PROVEN, to evaluate their value;
• Read laterally;
• Consider impact factors;
• Actively seek out diverse perspectives and, specifically, under-represented voices.

In the next sections of this module, we will examine these strategies more closely.

Basic source evaluation using set criteria

In order to ensure the sources you use are credible, you must, at the very minimum, evaluate them using some sort of criteria. Librarians at Centre recommend the PROVEN criteria:

Purpose - How and why the source was created

• Why does this information exist? To educate? To inform? To persuade? To sell? To entertain?
• Do the authors, publishers or sponsors state this purpose or try to disguise it?
• Who is the intended audience? The general public? Students? Experts?

 Relevance - The value of the source for your needs

• Is the type of source appropriate for how you plan to use it and for your assignment’s requirements? (ex. Is it valuable primary evidence? Is it a reliable secondary source?
• How useful is the information in this source, compared to other sources?
• Does it answer your question or support your argument? Does it add something new and important to your knowledge of the topic?
• How detailed is the information? Is it too general or too specific? Is it too basic or too advanced?

Objectivity - The reasonableness and completeness of the information

• Do the authors present the information thoroughly and professionally? Do they use strong, emotional, manipulative, or offensive language?
• Do the authors, publishers, or sponsors have a particular political, ideological, cultural, or religious point of view? Do they acknowledge this point of view, or try to disguise it?
• Does the source present fact, supported with empirical evidence, or opinion? Does it offer multiple points of view and critique other perspectives respectfully? Does it leave out, or mock, important facts or perspectives?

Verifiability -  The accuracy and truthfulness of the information

• Do the authors support their information with factual evidence? Do they cite or link to other sources? Can you verify the credibility of those sources? Can you find the original source of the information?
• What do other experts say about the topic? Can you verify the information in other credible sources?
• Does the source contradict itself, include false statements, or misrepresent other sources?
• Are there errors in spelling, punctuation, or grammar?

Expertise - The authority of the authors and the source

• What makes the authors, publishers, or sponsors of the source authorities on the topic? Do they have related education, or personal or professional experience? Are they affiliated with an educational institution or respected organization? Is their expertise acknowledged by other authorities on the topic? Do they provide an important alternative perspective?
• Do other sources cite this source?
• Has the source been reviewed by an editor or through peer review?
• Does the source provide contact information for the authors, publishers, and/or sponsors?

Newness - The age of the information

• Is your topic in an area that requires current information (such as science, technology, or current events), or could information found in older sources still be useful and valid?  
• When was the information in the source first published or posted? Are the references/links up to date?
• Are newer sources available that would add important information to your understanding of the topic?
• If this is an older source, is it a seminal/foundational source that has been cited numerous times (and is, therefore, still relevant as foundational information)?

Librarian Tip: There are many, similar "checklists" to evaluate the reliability of information that you may have learned, for example, the CRAAP test. These lists are deceptively easy. They appear to be a quick set of criteria you can check off easily. But how do you know if the author of a source is an expert? How do you know the purpose of a publication? How do you verify the evidence used in an article? Those are quite complicated tasks and require a practice called "Lateral Reading."

Demonstrate your research skills

Earn the Critical Evaluation - PROVEN Credential

We have discussed strategies to evaluate the information you gather as you research. Demonstrate you can implement these strategies by submitting a completed PROVEN analysis worksheet using the link provided below.

What is lateral reading?

How do you normally read an article? Most people read sequentially - they start at the first word of the article and read through to the end. Skilled academic researchers do not read articles sequentially. Instead they skim the introduction, then jump to the conclusion and results section at the end, before delving into the methodology and literature review sections, in the middle, for example. But studies have shown that the most critical reader uses a technique called lateral reading. The lateral reading concept and the term itself arose from research conducted by the Stanford History Education Group (SHEG). Lateral reading means that:

• before getting too committed to a source, you leave the source and use other known-credible sites to dig deep into the credentials of its author and any sponsoring organizations. You don't just rely on the author credential summary provided by the article or website;
• you also leave the source to verify any evidence it cites. You don't just assume the citation or link is accurate (the author may have misinterpreted or misrepresented or even fabricated the source).

Watch this video for more information:

Why is lateral reading important?

Let's look at the examples from the video in more detail. In the sections below, you will find screenshots from the two sources mentioned in the video. First, we will examine why they might easily pass a cursory evaluation. Then, we will use the sections of the article highlighted in red boxes and numbered with red numbers, to discuss how lateral reading helps us evaluate these sources more thoroughly.

Biased web resources

First, we'll examine this report found on the website of the American Council on Science and Health. Scan the image below and pay special attention to the red-highlighted sections:

Cursory evaluation If we only evaluated this site using a checklist criteria, like PROVEN, and we never left the page that contains this report, it would look pretty credible and relevant if our research on microplastics led us to be interested in PFOAs. The report was edited by a medical doctor (#3); the website says the sponsoring organization is dedicated to science, not junk science; the purpose of the website is stated to be education (#2); and empirical data is cited throughout the report (#4). A cursory evaluation might raise only one red flag - the "Donate" button (#1).

Lateral reading More thorough evaluation through lateral reading encourages us to not assume the author, organization and evidence in the report is credible, or to simply trust the information on the webpage "About Us" page. Instead, we should leave this page and thoroughly investigate each. When we do, we learn:

• If we Google the American Council on Science and Health (#2), the results show there is some controversy surrounding it. It is an openly right-leaning organization and is purportedly pro-industry, funded by big businesses. As such, the fact that its scientific reports publish pro-industry, rather than pro-environment, findings may represent bias.
• If we Google the editor of this report, Gilbert Ross (#3), the results show he lost his medical license after a felony conviction in a Medicaid fraud scheme.
• And if we look at the conclusions in this report (#4), we see that they do not align with the prevailing conclusions about PFOAs by other major scientific organizations.

Lateral reading demonstrates that resources published by this organization should be used with an awareness of their potential bias.

Predatory Journals

Few critical thinkers would be surprised that they need to be cautious of potentially biased web resources, such as the sample above, but what about scholarly articles? Shouldn't they be safe? Unfortunately, if you find an article online (on Google Scholar, for example), rather than in a library database (where all sources are already vetted), you must investigate the source yourself. Google Scholar makes no effort to ensure the results they return are truly peer-reviewed, rather than published in a predatory journal. A predatory journal is one that does little or no peer-review and publishes any article, provided the author pays the acceptance fee. Let's look at an example:

Cursory evaluation If we found this article, about upcycling/recycling to reduce microplastics, on Google Scholar, and evaluated it using solely list criteria, like PROVEN, we would probably find it to be appropriate for use in a college research paper: its authors are associated with a university (#4); it appears to use the format of a traditional scholarly article, with an introduction, literature review, findings, and conclusion (#1a, b, c); and it has numerous citations (#1d). Again, we have a red flag with the "Pay Fees" button (#2). The biggest red flag, without leaving the page to laterally read, are the "submitted" and "acceptance" dates (#3). The paper was accepted 15 days after it was submitted. Most valid peer-review processes take months. Very fast acceptance rates are a sign that a journal is doing little or no true peer-review. Combined with the "fees" button, we must suspect this is a predatory journal. We need to read laterally to learn more;

Lateral reading When we open a new tab and investigate this article:

• If we Google the names of the authors of these papers (#4), we learn that one was an undergraduate when this paper was written and the other has very few publications and few citations on those publications. Neither of these facts discredit the authors. They may be upcoming, important voices that should be amplified. But we must evaluate this article more.
• If we Google the journal name (#5), we learn it is, indeed, a confirmed predatory journal on several predatory journal warning lists. Therefore, it is unlikely this paper received valid peer-review. If the paper requirements for your class require peer-review, this paper might deceive you, since it follows the traditional scholarly format, but it is not acceptable since it was not peer-reviewed.

To learn more about Predatory Publishers and find tools to help you identify them, refer to our Predatory Publishing Guide.

 The lesson to be learned from these two examples is: do not be fooled by appearances. Professional appearing websites, claims of non-partisanship and a scholarly facade are not evidence of credibility. When you find a source online, you must work to verify its credibility by opening another browser tab to research:

• the background of the author
• any biases and funding of the publisher/organization
• the links or citations in the article, to confirm they have not been misrepresented

 Librarian tip: If you use resources you find in the library databases, those resources are fully vetted already. Use the databases!

Demonstrate your research skills

Earn the Critical Evaluation - Lateral Reading Credential

We have discussed strategies to evaluate the information you gather as you research. Demonstrate you can implement these strategies by submitting a completed Lateral Reading analysis worksheet using the link provided below.

Using impact factor to evaluate sources

In addition to deeply investigating the credentials of an author and any sponsoring organizations and ensuring evidence isn't misinterpreted or misrepresented, many researchers check the impact factor of a journal, author or article. Impact factors, in general, measure how often journals, authors or articles are cited. If a scholar cites an article, that implies that article has "impacted" their research in some way and that it is somehow valuable. There are three types of impact factor you might consider.

Journal-level impact factor

Journal-level impact factor reflects the number of citations of articles published in a journal. It is often calculated on the mean number of citations per article in a given time period. Journal-level impact factor can help distinguish credible journals from predatory journals (though predatory journals often manipulate or falsify their impact factor, so don't trust a self-reported journal impact factor). Common reputable journal-level impact measures are Journal Citation Reports, Google Scholar Metrics, and Eigenfactor. At Centre, you can use Web of Science to view a journal's Journal Citation Indicator. To do so, follow these steps:

• click on the journal name link in a given reference;
• the resulting pop-up displays the Journal Citation Indicator

Watch this video for more information about finding and interpreting the Journal Citation Indicator in Web of Science:

Author-level impact factor

Author-level impact factor reflects the number of publications and citations of the publications written by a specific author (usually calculated using h-index). You can use Web of Science to learn the h-index of an author by following these steps:

• Perform an author search
• Create a citation report for that author.
• The h-index will be listed in the report.

Watch this video to learn how to use Web of Science to find and interpret an author's h-index:

Article-level impact factor

Article-level impact factor reflects the number of citations of a specific single article. You can see the number of times an article has been cited in many databases and in Google Scholar. Here are two examples:

Web of Science:

Google Scholar:

Beware of the fact that the use of impact factors is controversial because, amongst other reasons, the metric can be easily manipulated, there is no direct correlation between the impact factor and the quality of the science in journals, and impact factor amplifies certain voices while discounting others, possibly leading valuable research to be "lost in the algorithm.". Despite these concerns, impact factors, especially journal impact factors, are heavily used in academia to evaluate scholars' work.

Intentionally seek diverse perspectives

It is a challenge for scholars doing research in many fields to search for and include marginalized voices within their work. Such voices include Black, Indigenous, and People of Color (BIPOC), LGBTQ+ folx, people with disabilities, people of lower socioeconomic status, people who are neurodiverse, and all marginalized genders and their intersections. Currently and historically, barriers exist that block groups from participating in the scholarly conversation.

Understand the problem

It is important to be aware of and acknowledge the fact that, historically, scholarship has suffered from a variety of forms of structural racism. Most published research is not representative of the majority of populations because it was conducted within WEIRD societies:

By this we mean many things.

At its most basic, WEIRD scholarship emerges from authors that, historically, were wealthy, white, intellectual men. Even today, a very low percentage of cited works are written by diverse authors, despite the fact that there are many more diverse scholars. Most editors are white and/or from Western nations, and 66-80% of peer reviewers are also white. This structure serves to marginalize non-white scholars.

WEIRD scholarship also benefits from the fact that public search algorithms (like Google's algorithm) are racist and library cataloging systems are racist. (Librarians, across the world and here at Centre, are working hard to correct this problem). This facilitates citation bias (the tendency for researchers to cite articles published in preferred, often predominately white-centered journals due to familiarity).

WIERD scholarship also refers to the fact that there is a history in the United States of research atrocities committed against non-white people (ex. Tuskegee Study, forced sterilization, intellectual property claims) and a lack of research that includes people who are neurodiverse and LGBTQ+ folx. Prior research exploitation is often used as an excuse to explain why so many clinical trials lack diversity in their study populations.

This is intended to be a very concise summary of a very large and complex problem. You are strongly advised to learn more about this problem. We suggest you might begin by consulting these excellent, and much more in-depth, guides:

Conducting research through an anti-racism lens

Multiply Marginalized & Underrepresented Scholars

And by watching this video:

Be part of the solution

What should you do to help solve the problem of structural racism in research? Here is a very non-exhaustive list of strategies you should employ in all your research:

1. Make a conscious effort to reject citation bias and include diverse voices when citing secondary sources:

• Search demographic specific databases (ex. Women's Studies Archive , African History and Culture, 1540-1921)   
• Find specific journals for marginalized groups (ex. LGBTQ Policy Journal)
• Research professional associations/organizations that are entirely focused on diverse groups (ex. Southern Poverty Law Center)
• Explore Digital Humanities projects (like Pathways to Diversity)
• Locate public archive collections focused on diverse groups
• Learn how the library catalog/database describes certain populations in their Subject Headings. Also, think of multiple ways to describe the population you are looking for
• Search for lists of authors from marginalized communities in your field 

2. When designing your research methodology, utilize methods that de-center whiteness and work to achieve racial equity throughout the data life cycle:

• Planning - Include diverse perspectives (such as community members with lived experiences and agency staff who understand the data) on planning committees
• Data collection - use diverse community resources to plan how to gather and secure data in a way that is inclusive
• Data access - Use a multi-tiered approach to data access to help ensure transparency and allow as much access to data as possible while still minimizing harm to participants.
• Data algorithms - employ the five guidelines from Fairness, Accountability, and Transparency in Machine Learning can be used as a starting place to inform the development and use of algorithmic tools that are more inclusive.
• Data analysis - bring multiple perspectives to the interpretation of the data.
• Reporting and disseminating - Create a range of products, using a range of language (specialized language to language for non-specialists) to communicate findings across a wide variety of audiences via both online and offline methods of dissemination and report data in an actionable form to improve the lives of those represented in the data. Include stories as a complement to quantitative findings in order to better contextualize the lived experience represented by the numbers

Overall, be sure to question traditional notions of how authority is conferred and where authority is found. Recognize the value of diverse ideas and worldviews.

Positionality statement

This section of this credential was developed by three librarians:Fernando Gonzalez, Karoline Manny and Jazmine Wilson. We are deeply indebted to the authors of the information in the links cited in this guide. We welcome contributions to and critiques of the information in this guide so that this living document may continue to evolve and improve. We encourage others to build on and evolve the conversation surrounding this very important topic.